amadan

carlos / platform Public

email events: keep what happened to each message, and `carlos email events` to read it

SES already publishes Send, Delivery, Bounce, Complaint, Reject and
DeliveryDelay to the deployment's SNS topic, and the ingest already
verifies every notification — but all it kept was a per-day count. The
message id reached mail.ParseEvent and was dropped there, along with who
the message went to and what the receiving server answered. A member
could see that something bounced today and had no way to find out what.

ParseEvent now reads the rest of the notification: the destination, the
automatic ses:caller-identity tag, and each type's own recipient block —
bounced, complained-about, delivered or delayed addresses with the
enhanced status code and the far end's own diagnostic. Only the block the
event's type names is read, and every kept string is stripped of control
characters and bounded: the notification is signed by SNS, but the text
in it was written by whichever mail server answered.

The console keeps it in mail_events, one row per recipient per
notification, keyed on SNS's own MessageId so a notification delivered
twice is one row. Recipient addresses are somebody's guests or customers,
so the table is a 30-day window: rows past it are never returned, and are
pruned at most hourly from the ingest path.

Attribution is narrower than the counts' where SES allows. CARLOS mints
one IAM identity per app (carlos-mail-<acct>-<app>), and SES reports the
sending identity in its automatic ses:caller-identity tag;
mail.AppOfCaller reads the app back out. (Not yet seen on a live event
from this deployment — the fallback below is what happens if it is
absent or shaped differently.) When that app is on
the matching domain, the row goes to it alone, so two apps sharing a From
domain do not see each other's recipients. When SES names nobody, or
somebody who is not one of those apps, the row falls back to every app on
the domain, exactly as the counts do, and says so (by_sender = 0) — which
is the distinction anything that later hands a row to an app will need.
The counts themselves are untouched: they enforce the cap, and changing
what they measure is not this change's business.

GET /api/cli/apps/{account}/{app}/email/events reads the log, Admin-only
because it names the people an app wrote to, 404 on a deployment with no
email like every door in the family, and deliberately not dependent on
the email record existing: thirty days of history are exactly what you
want after a teardown. `carlos email events` prints it newest first — what
happened, to whom, which message — with the receiving server's answer
beneath each line, and takes --since, --message, --to, -n and --json.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bill Horsman pushed by bill@keymail.dev 819e7cb8da9c25c0c6112f00fc811eb659d274d0 parent d9f67f2
13 files changed, +1161 −13
  • docs/site/cli.md +17 −1
  • internal/console/apimail.go +67 −0
  • internal/console/apimail_test.go +53 −0
  • internal/console/mailevents.go +153 −0
  • internal/console/mailevents_test.go +190 −0
  • internal/console/sesevents.go +37 −2
  • internal/console/sesevents_test.go +1 −1
  • internal/console/store.go +43 −0
  • internal/mail/events.go +181 −8
  • internal/mail/events_test.go +155 −1
  • internal/mail/policy.go +33 −0
  • mail_cli.go +177 −0
  • mail_cli_test.go +54 −0
diff --git a/docs/site/cli.md b/docs/site/cli.md
index 0963b364d..79e2b6185 100644
--- a/docs/site/cli.md
+++ b/docs/site/cli.md
@@ -866,7 +866,23 @@ it proves that mail leaves the building rather than that the console believes
it should.
`status` shows every domain's per-region verification state, the day's count
-against the cap, and whether sending is paused. `credentials create` mints a
+against the cap, and whether sending is paused.
+
+`events` is what happened to each message: delivered, bounced, complained
+about or delayed, newest first, with what the receiving server said — the
+line that tells you whether a bounce was a mistyped address or a full
+mailbox. `--message` narrows it to one message by the id SES gave it when it
+was sent, `--to` to one recipient, and `--since` to a window. Events are kept
+for 30 days. Where SES names the credential that sent a message, its events
+belong to that app alone, so two apps sharing a sending domain do not see
+each other's recipients; where it does not, they go to every app on the
+domain, as the day's counts always have.
+
+```sh
+carlos email events --app hello --since 24h
+carlos email events --app hello --to guest@example.com
+```
+ `credentials create` mints a
standalone SMTP credential for something not running on CARLOS — a laptop, a
cron box — printed once on that command's output and nowhere else. `rotate`
mints a fresh delivered credential and leaves the old one live until you run
diff --git a/internal/console/apimail.go b/internal/console/apimail.go
index 59359f671..bcc6ed024 100644
--- a/internal/console/apimail.go
+++ b/internal/console/apimail.go
@@ -55,6 +55,7 @@ import (
"log"
"net/http"
"sort"
+ "strconv"
"strings"
"time"
@@ -462,6 +463,10 @@ func (s *Server) mailWalkReply(ctx context.Context, w http.ResponseWriter, acct
// .../store is: the status reply names the IAM user, the key id and the
// verified domains — the estate, not just the app.
//
+// GET .../email/events is Admin too, and for a different reason: it names
+// the people the app wrote to. A credential that may read an app's routes
+// or deploy it has no need of its recipients' addresses.
+//
// The three CLOSED doors (svcrole.Set{} — closed to every role, admin
// included) are each closed for their own reason:
//
@@ -489,6 +494,7 @@ func (s *Server) mailWalkReply(ctx context.Context, w http.ResponseWriter, acct
func (s *Server) registerEmailRoutes() {
s.cliRoute("POST /api/cli/apps/{account}/{app}/email", svcrole.Allow(svcrole.Admin), s.handleAPIEmailDeclare)
s.cliRoute("GET /api/cli/apps/{account}/{app}/email", svcrole.Allow(svcrole.Admin), s.handleAPIEmailStatus)
+ s.cliRoute("GET /api/cli/apps/{account}/{app}/email/events", svcrole.Allow(svcrole.Admin), s.handleAPIEmailEvents)
s.cliRoute("POST /api/cli/apps/{account}/{app}/email/domains", svcrole.Allow(svcrole.Admin), s.handleAPIEmailDomainEnsure)
s.cliRoute("POST /api/cli/apps/{account}/{app}/email/domains/{domain}/verify", svcrole.Allow(svcrole.Admin), s.handleAPIEmailDomainVerify)
s.cliRoute("POST /api/cli/apps/{account}/{app}/email/enable", svcrole.Allow(svcrole.Admin), s.handleAPIEmailEnable)
@@ -648,6 +654,67 @@ func (s *Server) handleAPIEmailStatus(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, out)
}
+// ---- events ----------------------------------------------------------------
+
+// handleAPIEmailEvents answers `carlos email events`: what happened to the
+// messages the app sent — delivered, bounced, complained about, delayed —
+// newest first, from the per-message log (mailevents.go). Not rate limited,
+// like every read door.
+//
+// It deliberately does NOT require the email record to exist: an app whose
+// email was torn down still has thirty days of history, and "what happened
+// to the last thing we sent" is exactly the question after a teardown. Nor
+// does it need the bucket — the log is the console's own table. What it
+// keeps from the family is the provisioner gate: a deployment with no email
+// answers 404 here as on every other email door.
+//
+// Query parameters, all optional: since (RFC 3339), message (an SES message
+// id), to (an address), limit (1–1000, default 100).
+func (s *Server) handleAPIEmailEvents(w http.ResponseWriter, r *http.Request) {
+ _, acct, app, ok := s.apiAccountApp(w, r)
+ if !ok {
+ return
+ }
+ if !s.mailProvisionerGate(w) {
+ return
+ }
+ q := MailEventQuery{
+ MessageID: strings.TrimSpace(r.URL.Query().Get("message")),
+ Recipient: strings.TrimSpace(r.URL.Query().Get("to")),
+ }
+ if raw := strings.TrimSpace(r.URL.Query().Get("since")); raw != "" {
+ since, err := time.Parse(time.RFC3339, raw)
+ if err != nil {
+ jsonError(w, http.StatusBadRequest, "since must be an RFC 3339 time, like 2026-09-15T08:00:00Z")
+ return
+ }
+ q.Since = since
+ }
+ if raw := strings.TrimSpace(r.URL.Query().Get("limit")); raw != "" {
+ n, err := strconv.Atoi(raw)
+ if err != nil || n < 1 || n > mailEventsMaxLimit {
+ jsonError(w, http.StatusBadRequest, fmt.Sprintf("limit must be a whole number from 1 to %d", mailEventsMaxLimit))
+ return
+ }
+ q.Limit = n
+ }
+ ctx, cancel := context.WithTimeout(r.Context(), mailTimeout)
+ defer cancel()
+ events, err := s.store.MailEvents(ctx, acct.Sqid, app, q, time.Now())
+ if err != nil {
+ log.Printf("console: api email events %s/%s: %v", acct.Sqid, app, err)
+ jsonError(w, http.StatusBadGateway, "could not read the email log — try again")
+ return
+ }
+ if events == nil {
+ events = []MailEvent{}
+ }
+ writeJSON(w, http.StatusOK, map[string]any{
+ "events": events,
+ "retention_days": int(mailEventsRetention / (24 * time.Hour)),
+ })
+}
+
// ---- domains ----------------------------------------------------------------
// handleAPIEmailDomainEnsure answers `carlos email domains add`: bring a
diff --git a/internal/console/apimail_test.go b/internal/console/apimail_test.go
index 2322f56d5..dabb8b2c3 100644
--- a/internal/console/apimail_test.go
+++ b/internal/console/apimail_test.go
@@ -163,6 +163,7 @@ func TestEmailDoorsAre404ForNonMembers(t *testing.T) {
call func(base, bearer string) (int, string)
}{
{"GET status", func(base, bearer string) (int, string) { return getRaw(t, base, bearer) }},
+ {"GET events", func(base, bearer string) (int, string) { return getRaw(t, base+"/events", bearer) }},
{"POST declare", func(base, bearer string) (int, string) {
return postRaw(t, base, map[string]any{"from": "hello@example.com"}, bearer)
}},
@@ -224,6 +225,7 @@ func TestEmailDoorsAre404WithNoProvisioner(t *testing.T) {
do func() (int, string)
}{
{"GET status", func() (int, string) { return getRaw(t, emailURL(fx, "titogo"), token) }},
+ {"GET events", func() (int, string) { return getRaw(t, emailURL(fx, "titogo")+"/events", token) }},
{"POST declare", func() (int, string) {
return postRaw(t, emailURL(fx, "titogo"), map[string]any{"from": "hello@example.com"}, token)
}},
@@ -1083,3 +1085,54 @@ func TestAPIEmailDomainEnsureRefusesADomainClaimedByAnotherAccount(t *testing.T)
t.Fatalf("refusal promised a retry would help, but this refusal is permanent: %q", msg)
}
}
+
+// TestAPIEmailEventsReadsTheLog: the door `carlos email events` calls hands
+// back the app's per-message log, narrowed by what the query asks, and
+// refuses a query it cannot read rather than guessing at it.
+func TestAPIEmailEventsReadsTheLog(t *testing.T) {
+ fx, _, _, token := emailArmed(t, "code-email-events")
+ app := "titogo"
+ ctx := context.Background()
+ now := time.Now()
+ for _, c := range []struct{ to, id, typ string }{
+ {"ann@example.com", "m-ann", mail.EventDelivery}, {"bob@example.com", "m-bob", mail.EventBounce},
+ } {
+ ev := mail.Event{Type: c.typ, MessageID: c.id, At: now, Outcomes: []mail.Outcome{{Recipient: c.to}}}
+ if err := fx.st.RecordMailMessageEvent(ctx, fx.sqid, app, "sns-"+c.id, ev, true, now); err != nil {
+ t.Fatal(err)
+ }
+ }
+
+ status, out := getJSON(t, emailURL(fx, app)+"/events", token)
+ if status != http.StatusOK {
+ t.Fatalf("status = %d %v", status, out)
+ }
+ events, _ := out["events"].([]any)
+ if len(events) != 2 {
+ t.Fatalf("events = %v, want both", out["events"])
+ }
+ if days, _ := out["retention_days"].(float64); days != 30 {
+ t.Errorf("retention_days = %v, want 30 — the reader should be told how far back it can look", out["retention_days"])
+ }
+
+ status, out = getJSON(t, emailURL(fx, app)+"/events?to=BOB%40example.com", token)
+ events, _ = out["events"].([]any)
+ if status != http.StatusOK || len(events) != 1 {
+ t.Fatalf("narrowed to bob: %d %v", status, out)
+ }
+ if first, _ := events[0].(map[string]any); first["message_id"] != "m-bob" || first["type"] != mail.EventBounce {
+ t.Errorf("row = %v", first)
+ }
+
+ // An app with nothing in its log is an empty list, not a null.
+ if status, body := getRaw(t, emailURL(fx, app)+"/events?message=nothing-by-this-id", token); status != http.StatusOK ||
+ !strings.Contains(body, `"events":[]`) {
+ t.Errorf("an empty answer = %d %s", status, body)
+ }
+
+ for _, bad := range []string{"since=yesterday", "limit=0", "limit=100000", "limit=many"} {
+ if status, body := getRaw(t, emailURL(fx, app)+"/events?"+bad, token); status != http.StatusBadRequest {
+ t.Errorf("%s = %d %s, want 400", bad, status, body)
+ }
+ }
+}
diff --git a/internal/console/mailevents.go b/internal/console/mailevents.go
new file mode 100644
index 000000000..fb487a424
--- /dev/null
+++ b/internal/console/mailevents.go
@@ -0,0 +1,153 @@
+package console
+
+// The per-message email log: what happened to each message an app sent, to
+// whom, and what the receiving server said. The counts in mailcounts.go are
+// the abuse signal and stay exactly as they were; this sits beside them and
+// is fed from the same verified SES notifications, one row per recipient
+// per notification.
+
+import (
+ "context"
+ "database/sql"
+ "strings"
+ "time"
+
+ "github.com/carlosframework/platform/internal/mail"
+)
+
+// mailEventsRetention is how long a row is kept. The log carries recipient
+// addresses — somebody's guests, customers, users — so it is a window, not
+// an archive: long enough to answer "did that email last week arrive", no
+// longer.
+const mailEventsRetention = 30 * 24 * time.Hour
+
+// mailEventsPruneEvery bounds how often the prune runs. It is one indexed
+// DELETE, but it would otherwise run once per notification, and an hour's
+// slack on a thirty-day window changes nothing anybody can see.
+const mailEventsPruneEvery = time.Hour
+
+// MailEvent is one row of the log.
+type MailEvent struct {
+ MessageID string `json:"message_id"`
+ Type string `json:"type"`
+ Recipient string `json:"recipient"`
+ Status string `json:"status,omitempty"`
+ Diagnostic string `json:"diagnostic,omitempty"`
+ Detail string `json:"detail,omitempty"`
+ BySender bool `json:"by_sender"`
+ At time.Time `json:"at"`
+}
+
+// RecordMailMessageEvent keeps one notification for one app: a row per
+// recipient it concerns. notice is SNS's MessageId for the notification,
+// and a notification SNS delivers twice writes nothing the second time.
+// bySender says whether SES itself named this app as the sender, rather
+// than the app being one of several that share the sending domain.
+func (s *Store) RecordMailMessageEvent(ctx context.Context, acct, app, notice string, ev mail.Event, bySender bool, now time.Time) error {
+ at := ev.At
+ if at.IsZero() {
+ at = now
+ }
+ if notice == "" {
+ // A notification with no SNS id of its own — nothing SNS sends, but
+ // the recording path is also reachable directly. Key it on what it
+ // says instead, so a repeat still collapses.
+ notice = ev.MessageID + "/" + ev.Type + "/" + at.UTC().Format(time.RFC3339Nano)
+ }
+ sender := 0
+ if bySender {
+ sender = 1
+ }
+ detail := ev.Detail.Summary()
+ for _, o := range ev.Recipients() {
+ if _, err := s.db.ExecContext(ctx, `INSERT OR IGNORE INTO mail_events
+ (account, app, notice, recipient, message_id, type, status, diagnostic, detail, by_sender, at, received_at)
+VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+ acct, app, notice, o.Recipient, ev.MessageID, ev.Type, o.Status, o.Diagnostic, detail, sender,
+ at.Unix(), now.Unix()); err != nil {
+ return err
+ }
+ }
+ return s.pruneMailEvents(ctx, now)
+}
+
+// pruneMailEvents trims the log to its retention window, at most once per
+// mailEventsPruneEvery. The stamp is taken before the DELETE runs, so two
+// notifications arriving together do not both prune.
+func (s *Store) pruneMailEvents(ctx context.Context, now time.Time) error {
+ last := s.mailEventsPruned.Load()
+ if last != 0 && now.Unix()-last < int64(mailEventsPruneEvery/time.Second) {
+ return nil
+ }
+ if !s.mailEventsPruned.CompareAndSwap(last, now.Unix()) {
+ return nil
+ }
+ _, err := s.db.ExecContext(ctx, `DELETE FROM mail_events WHERE received_at < ?`, now.Add(-mailEventsRetention).Unix())
+ return err
+}
+
+// MailEventQuery narrows a read of the log. The zero value is the newest
+// mailEventsDefaultLimit rows of the whole window.
+type MailEventQuery struct {
+ Since time.Time // only rows SES timestamped at or after this
+ MessageID string // only this message
+ Recipient string // only this address, case-insensitively
+ Limit int
+}
+
+const (
+ mailEventsDefaultLimit = 100
+ mailEventsMaxLimit = 1000
+)
+
+// MailEvents reads one app's log, newest first. A row older than the
+// retention window is never returned, whether or not the prune has reached
+// it yet — what a member is told is kept for thirty days has to be what
+// they can see.
+func (s *Store) MailEvents(ctx context.Context, acct, app string, q MailEventQuery, now time.Time) ([]MailEvent, error) {
+ limit := q.Limit
+ if limit <= 0 {
+ limit = mailEventsDefaultLimit
+ }
+ if limit > mailEventsMaxLimit {
+ limit = mailEventsMaxLimit
+ }
+ where := []string{"account = ?", "app = ?", "received_at >= ?"}
+ args := []any{acct, app, now.Add(-mailEventsRetention).Unix()}
+ if !q.Since.IsZero() {
+ where = append(where, "at >= ?")
+ args = append(args, q.Since.Unix())
+ }
+ if q.MessageID != "" {
+ where = append(where, "message_id = ?")
+ args = append(args, q.MessageID)
+ }
+ if q.Recipient != "" {
+ where = append(where, "recipient = ? COLLATE NOCASE")
+ args = append(args, q.Recipient)
+ }
+ args = append(args, limit)
+ rows, err := s.db.QueryContext(ctx, `SELECT message_id, type, recipient, status, diagnostic, detail, by_sender, at
+FROM mail_events WHERE `+strings.Join(where, " AND ")+`
+ORDER BY at DESC, received_at DESC, recipient LIMIT ?`, args...)
+ if err != nil {
+ return nil, err
+ }
+ defer rows.Close()
+ var out []MailEvent
+ for rows.Next() {
+ var e MailEvent
+ var at int64
+ var sender int
+ if err := rows.Scan(&e.MessageID, &e.Type, &e.Recipient, &e.Status, &e.Diagnostic, &e.Detail, &sender, &at); err != nil {
+ return nil, err
+ }
+ e.BySender = sender == 1
+ e.At = time.Unix(at, 0).UTC()
+ out = append(out, e)
+ }
+ if err := rows.Err(); err != nil && err != sql.ErrNoRows {
+ return nil, err
+ }
+ return out, nil
+}
diff --git a/internal/console/mailevents_test.go b/internal/console/mailevents_test.go
new file mode 100644
index 000000000..5bbd0eb4b
--- /dev/null
+++ b/internal/console/mailevents_test.go
@@ -0,0 +1,190 @@
+package console
+
+import (
+ "context"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/mail"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+func bounceTo(recipient, message string, at time.Time) mail.Event {
+ return mail.Event{Type: mail.EventBounce, Domain: "lgc.li", MessageID: message, At: at,
+ Outcomes: []mail.Outcome{{Recipient: recipient, Status: "5.1.1", Diagnostic: "550 5.1.1 no such user"}},
+ Detail: mail.EventDetail{BounceType: "Permanent", BounceSubType: "NoEmail"}}
+}
+
+// A bounce is kept as what it was: which message, to whom, and what the
+// receiving server answered — and a notification SNS delivers twice is
+// still one row.
+func TestTheMailLogKeepsEachRecipientsOutcomeOnce(t *testing.T) {
+ fx := newConsoleFixture(t)
+ ctx := context.Background()
+ now := time.Date(2026, 9, 15, 9, 0, 0, 0, time.UTC)
+ ev := bounceTo("ann@example.com", "0102-ann", now.Add(-time.Minute))
+
+ for i := 0; i < 2; i++ {
+ if err := fx.st.RecordMailMessageEvent(ctx, "beb", "jb-book", "sns-1", ev, true, now); err != nil {
+ t.Fatal(err)
+ }
+ }
+ got, err := fx.st.MailEvents(ctx, "beb", "jb-book", MailEventQuery{}, now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(got) != 1 {
+ t.Fatalf("rows = %d, want 1 — SNS redelivered one notification: %+v", len(got), got)
+ }
+ e := got[0]
+ if e.MessageID != "0102-ann" || e.Type != mail.EventBounce || e.Recipient != "ann@example.com" ||
+ e.Status != "5.1.1" || e.Detail != "Permanent/NoEmail" || !e.BySender {
+ t.Errorf("row = %+v", e)
+ }
+ if !strings.Contains(e.Diagnostic, "no such user") {
+ t.Errorf("the receiving server's answer was not kept: %q", e.Diagnostic)
+ }
+}
+
+func TestTheMailLogFindsAMessageOrAnAddress(t *testing.T) {
+ fx := newConsoleFixture(t)
+ ctx := context.Background()
+ now := time.Date(2026, 9, 15, 9, 0, 0, 0, time.UTC)
+ for i, c := range []struct{ to, id string }{
+ {"ann@example.com", "m-ann"}, {"bob@example.com", "m-bob"}, {"ann@example.com", "m-ann-2"},
+ } {
+ ev := bounceTo(c.to, c.id, now.Add(-time.Duration(3-i)*time.Hour))
+ if err := fx.st.RecordMailMessageEvent(ctx, "beb", "jb-book", "sns-"+c.id, ev, true, now); err != nil {
+ t.Fatal(err)
+ }
+ }
+ for _, c := range []struct {
+ name string
+ q MailEventQuery
+ want []string
+ }{
+ {"everything, newest first", MailEventQuery{}, []string{"m-ann-2", "m-bob", "m-ann"}},
+ {"one message", MailEventQuery{MessageID: "m-bob"}, []string{"m-bob"}},
+ {"one address, however it is typed", MailEventQuery{Recipient: "ANN@Example.com"}, []string{"m-ann-2", "m-ann"}},
+ {"since", MailEventQuery{Since: now.Add(-150 * time.Minute)}, []string{"m-ann-2", "m-bob"}},
+ {"a limit", MailEventQuery{Limit: 1}, []string{"m-ann-2"}},
+ } {
+ t.Run(c.name, func(t *testing.T) {
+ got, err := fx.st.MailEvents(ctx, "beb", "jb-book", c.q, now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var ids []string
+ for _, e := range got {
+ ids = append(ids, e.MessageID)
+ }
+ if strings.Join(ids, ",") != strings.Join(c.want, ",") {
+ t.Errorf("got %v, want %v", ids, c.want)
+ }
+ })
+ }
+ if other, _ := fx.st.MailEvents(ctx, "beb", "another-app", MailEventQuery{}, now); len(other) != 0 {
+ t.Errorf("another app read this app's log: %+v", other)
+ }
+}
+
+// The log holds recipients' addresses, so it is a window. A row past it is
+// neither shown nor kept.
+func TestTheMailLogForgetsAfterThirtyDays(t *testing.T) {
+ fx := newConsoleFixture(t)
+ ctx := context.Background()
+ then := time.Date(2026, 8, 1, 9, 0, 0, 0, time.UTC)
+ if err := fx.st.RecordMailMessageEvent(ctx, "beb", "jb-book", "sns-old", bounceTo("ann@example.com", "m-old", then), true, then); err != nil {
+ t.Fatal(err)
+ }
+ now := then.Add(mailEventsRetention + time.Hour)
+ if got, _ := fx.st.MailEvents(ctx, "beb", "jb-book", MailEventQuery{}, now); len(got) != 0 {
+ t.Errorf("a row past the window was shown: %+v", got)
+ }
+ // The next notification prunes it for good.
+ if err := fx.st.RecordMailMessageEvent(ctx, "beb", "jb-book", "sns-new", bounceTo("bob@example.com", "m-new", now), true, now); err != nil {
+ t.Fatal(err)
+ }
+ var n int
+ if err := fx.st.db.QueryRow(`SELECT count(*) FROM mail_events WHERE message_id = 'm-old'`).Scan(&n); err != nil {
+ t.Fatal(err)
+ }
+ if n != 0 {
+ t.Errorf("the old row is still in the table")
+ }
+}
+
+// Two apps can send from one domain. Their counts are attributed by the
+// domain, as they always were; the log is narrower wherever SES names the
+// sender, because a recipient's address is one app's business, not every
+// app's that happens to share a From domain.
+func TestTheMailLogGoesToTheAppSESSaysSentIt(t *testing.T) {
+ fx := newConsoleFixture(t)
+ ctx := context.Background()
+ now := time.Now()
+ for _, app := range []string{"jb-book", "jb-site"} {
+ if _, err := fx.st.CreateAppIn(OpsAccountID, app, "bill@tito.io", now); err != nil {
+ t.Fatal(err)
+ }
+ }
+ rec := &mail.DomainRecord{
+ Domain: "lgc.li", Kind: mail.KindCustom, Apps: []string{"jb-book", "jb-site"},
+ Regions: map[string]*mail.RegionState{"eu-west-1": {
+ IdentityARN: "arn:aws:ses:eu-west-1:1:identity/lgc.li", MailFrom: "mail-eu-west-1.lgc.li", Status: mail.StatusVerified,
+ }},
+ CreatedAt: now,
+ }
+ if _, err := mail.PutDomain(ctx, fx.rel, release.OpsAccount, rec, ""); err != nil {
+ t.Fatal(err)
+ }
+ logOf := func(app string) []MailEvent {
+ t.Helper()
+ got, err := fx.st.MailEvents(ctx, release.OpsAccount, app, MailEventQuery{}, time.Now())
+ if err != nil {
+ t.Fatal(err)
+ }
+ return got
+ }
+
+ named := bounceTo("ann@example.com", "m-named", now)
+ named.Caller = mail.UserName(release.OpsAccount, "jb-book")
+ fx.srv.recordAndEnforceMail(ctx, named, "sns-named")
+
+ if got := logOf("jb-book"); len(got) != 1 || !got[0].BySender {
+ t.Errorf("the app SES named should have the row, marked as its own: %+v", got)
+ }
+ if got := logOf("jb-site"); len(got) != 0 {
+ t.Errorf("the other app on the domain was shown another app's recipient: %+v", got)
+ }
+ day := now.UTC().Format("2006-01-02")
+ for _, app := range []string{"jb-book", "jb-site"} {
+ if c, _ := fx.st.MailCounts(ctx, release.OpsAccount, app, day); c.Bounces != 1 {
+ t.Errorf("%s bounces = %d — the counts' attribution is not this change's to narrow", app, c.Bounces)
+ }
+ }
+
+ // SES naming nobody — or somebody who is not one of these apps — falls
+ // back to the domain, and says so on the row.
+ for i, caller := range []string{"", "someone-else", mail.UserName(release.OpsAccount, "not-on-this-domain")} {
+ ev := bounceTo("bob@example.com", "m-unnamed", now)
+ ev.Caller = caller
+ fx.srv.recordAndEnforceMail(ctx, ev, "sns-unnamed-"+string(rune('a'+i)))
+ }
+ for _, app := range []string{"jb-book", "jb-site"} {
+ var unnamed []MailEvent
+ for _, e := range logOf(app) {
+ if e.MessageID == "m-unnamed" {
+ unnamed = append(unnamed, e)
+ }
+ }
+ if len(unnamed) != 3 {
+ t.Errorf("%s: %d domain-attributed rows, want 3", app, len(unnamed))
+ }
+ for _, e := range unnamed {
+ if e.BySender {
+ t.Errorf("%s: a row attributed by domain claims SES named the sender: %+v", app, e)
+ }
+ }
+ }
+}
diff --git a/internal/console/sesevents.go b/internal/console/sesevents.go
index 05fabef6e..489960a44 100644
--- a/internal/console/sesevents.go
+++ b/internal/console/sesevents.go
@@ -404,7 +404,7 @@ func (s *Server) handleSESEvents(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
return
}
- s.recordAndEnforceMail(r.Context(), ev)
+ s.recordAndEnforceMail(r.Context(), ev, env.MessageID)
w.WriteHeader(http.StatusOK)
default:
// An SES event type this deployment does not act on (SES has more
@@ -451,7 +451,15 @@ func domainMatchesEvent(d *mail.DomainRecord, domain string) bool {
// bounded number of accounts; this is the same shape ListDomains' own
// callers (mail.Teardown, mail.grantedDomains) already accept for a single
// account, one level up.
-func (s *Server) recordAndEnforceMail(ctx context.Context, ev mail.Event) {
+//
+// Each notification is also kept in the per-message log (mailevents.go),
+// under notice — SNS's own id for it. The log is attributed more narrowly
+// than the counts where SES allows: when the notification names the IAM
+// identity that sent it, and that identity is one of the apps on the
+// matching domain, the row goes to that app alone. The counts keep their
+// domain-wide attribution untouched — they enforce a cap, and changing what
+// they measure is not this change's business.
+func (s *Server) recordAndEnforceMail(ctx context.Context, ev mail.Event, notice string) {
accts, err := s.store.Accounts()
if err != nil {
log.Printf("console: ses events: listing accounts for %s: %v", ev.Domain, err)
@@ -479,7 +487,15 @@ func (s *Server) recordAndEnforceMail(ctx context.Context, ev mail.Event) {
if !domainMatchesEvent(d, ev.Domain) {
continue
}
+ sender := senderAmong(ev, a.Sqid, d.Apps)
for _, app := range d.Apps {
+ // The log first, and on its own terms: a failure here is
+ // logged and must not cost the count the cap relies on.
+ if sender == "" || sender == app {
+ if err := s.store.RecordMailMessageEvent(ctx, a.Sqid, app, notice, ev, sender == app, time.Now()); err != nil {
+ log.Printf("console: ses events: logging %s for %s/%s: %v", ev.Type, a.Sqid, app, err)
+ }
+ }
if err := s.store.RecordMailEvent(ctx, a.Sqid, app, day, ev.Type); err != nil {
log.Printf("console: ses events: recording %s for %s/%s: %v", ev.Type, a.Sqid, app, err)
continue
@@ -494,6 +510,25 @@ func (s *Server) recordAndEnforceMail(ctx context.Context, ev mail.Event) {
}
}
+// senderAmong names the app SES says sent ev, if it is one of apps in
+// account acct — and "" when SES named no identity, named one that is not
+// CARLOS's, or named an app in another account or not on this domain. ""
+// is the fallback to attribution by domain, which is what every event had
+// before SES's sender was read at all: no narrower, and never pointed at an
+// app the domain does not list.
+func senderAmong(ev mail.Event, acct string, apps []string) string {
+ callerAcct, callerApp, ok := mail.AppOfCaller(ev.Caller)
+ if !ok || callerAcct != acct {
+ return ""
+ }
+ for _, app := range apps {
+ if app == callerApp {
+ return app
+ }
+ }
+ return ""
+}
+
// confirmSNSSubscription GETs the verified, host-checked SubscribeURL. It is
// reached only after the enclosing message's signature has verified AND
// SubscribeURL has passed validSigningCertURL — the signature proves AWS
diff --git a/internal/console/sesevents_test.go b/internal/console/sesevents_test.go
index d02c32a5f..abb90406e 100644
--- a/internal/console/sesevents_test.go
+++ b/internal/console/sesevents_test.go
@@ -772,7 +772,7 @@ func TestRecordAndEnforceMailAttributesTheRewrittenMailFromEnvelope(t *testing.T
}
ev := mail.Event{Type: mail.EventSend, Domain: "mail-eu-west-1.harbourmail.bag.oncarlos.com", At: now}
- fx.srv.recordAndEnforceMail(ctx, ev)
+ fx.srv.recordAndEnforceMail(ctx, ev, "")
day := now.UTC().Format("2006-01-02")
counts, err := fx.srv.store.MailCounts(ctx, release.OpsAccount, "harbourmail", day)
diff --git a/internal/console/store.go b/internal/console/store.go
index a1099538a..575adebfb 100644
--- a/internal/console/store.go
+++ b/internal/console/store.go
@@ -19,6 +19,7 @@ import (
"os"
"path/filepath"
"strings"
+ "sync/atomic"
"time"
_ "modernc.org/sqlite"
@@ -37,6 +38,12 @@ const (
// deployment bucket.
type Store struct {
db *sql.DB
+
+ // mailEventsPruned is when mail_events was last trimmed to its
+ // retention window, as unix seconds (mailevents.go). Zero means not yet
+ // this process, which is what makes the first event after a restart
+ // prune.
+ mailEventsPruned atomic.Int64
}
// Open opens (creating if needed) the console DB at path.
@@ -197,6 +204,42 @@ func (s *Store) migrate() error {
rejects INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (account, app, day)
);`,
+ // What happened to each message, per (account, app) — one row per
+ // recipient per SES notification (mailevents.go). The counts above
+ // say how many bounced today; this says which message, to whom, and
+ // what the receiving server answered, so that a member can find out
+ // and an app can be told. Keyed like mail_counts, by the account
+ // sqid, for the same reason.
+ //
+ // notice is SNS's own MessageId for the notification: SNS delivers
+ // at least once, and a notification delivered twice is one row, not
+ // two. by_sender records HOW the row was attributed — 1 when SES
+ // named this app's own sending identity, 0 when the sending domain
+ // was all there was to go on and every app sharing it got a copy.
+ // It is kept because the difference matters to anything that hands
+ // a row on: only a row SES attributed to the app itself is that
+ // app's to be told about.
+ //
+ // Recipient addresses are personal data, so the table is bounded in
+ // time (mailEventsRetention) rather than kept for ever.
+ `CREATE TABLE IF NOT EXISTS mail_events (
+ account TEXT NOT NULL,
+ app TEXT NOT NULL,
+ notice TEXT NOT NULL,
+ recipient TEXT NOT NULL,
+ message_id TEXT NOT NULL,
+ type TEXT NOT NULL,
+ status TEXT NOT NULL DEFAULT '',
+ diagnostic TEXT NOT NULL DEFAULT '',
+ detail TEXT NOT NULL DEFAULT '',
+ by_sender INTEGER NOT NULL DEFAULT 0,
+ at INTEGER NOT NULL,
+ received_at INTEGER NOT NULL,
+ PRIMARY KEY (account, app, notice, recipient)
+);`,
+ `CREATE INDEX IF NOT EXISTS mail_events_by_time ON mail_events (account, app, at);`,
+ `CREATE INDEX IF NOT EXISTS mail_events_by_message ON mail_events (account, app, message_id);`,
+ `CREATE INDEX IF NOT EXISTS mail_events_by_age ON mail_events (received_at);`,
} {
if _, err := s.db.Exec(stmt); err != nil {
return err
diff --git a/internal/mail/events.go b/internal/mail/events.go
index 959d5f4d9..e7a9694d3 100644
--- a/internal/mail/events.go
+++ b/internal/mail/events.go
@@ -8,14 +8,16 @@ import (
"fmt"
"strings"
"time"
+ "unicode/utf8"
)
const (
- EventSend = "Send"
- EventBounce = "Bounce"
- EventComplaint = "Complaint"
- EventReject = "Reject"
- EventDelivery = "Delivery"
+ EventSend = "Send"
+ EventBounce = "Bounce"
+ EventComplaint = "Complaint"
+ EventReject = "Reject"
+ EventDelivery = "Delivery"
+ EventDeliveryDelay = "DeliveryDelay"
)
// Event is the part of an SES notification this package acts on.
@@ -25,6 +27,72 @@ type Event struct {
From string
MessageID string
At time.Time
+
+ // Caller is the IAM identity SES says sent the message — the automatic
+ // "ses:caller-identity" tag. CARLOS mints one per app (UserName), so
+ // where the sending domain can only name every app that shares it, this
+ // names the one app that actually sent. Empty when SES did not say;
+ // AppOfCaller reads it back.
+ Caller string
+ // Destination is who the message was addressed to (mail.destination).
+ Destination []string
+ // Outcomes is what this event says about each recipient it concerns:
+ // the bounced, complained-about, delivered or delayed addresses, each
+ // with what the far end said. A Send or a Reject is about the whole
+ // message and carries none.
+ Outcomes []Outcome
+ // Detail is the event-wide explanation, empty where the type has none.
+ Detail EventDetail
+}
+
+// Outcome is one recipient's part in an event.
+type Outcome struct {
+ Recipient string
+ // Status is the enhanced status code the receiving server gave, e.g.
+ // "5.1.1". Empty on a delivery and a complaint.
+ Status string
+ // Diagnostic is the receiving server's own words: the bounce or delay
+ // diagnostic, or the SMTP response it accepted a delivery with.
+ Diagnostic string
+}
+
+// EventDetail is what an event says about the message as a whole.
+type EventDetail struct {
+ BounceType string // Permanent, Transient, Undetermined
+ BounceSubType string // General, NoEmail, Suppressed, MailboxFull, ...
+ ComplaintType string // abuse, fraud, not-spam, ... when the provider said
+ RejectReason string // e.g. "Bad content"
+ DelayType string // e.g. MailboxFull, TransientCommunicationFailure
+}
+
+// The longest a field from an event is kept at. An address is at most 254
+// octets by RFC 5321; a diagnostic is the far end's free text and has no
+// limit of its own, so it gets one here — enough for any real explanation,
+// not enough for a receiving server to fill a table with.
+const (
+ maxAddressLen = 254
+ maxDiagnosticLen = 1000
+ maxDetailLen = 64
+)
+
+// cleanField drops control characters and bounds the length. Every string
+// that comes out of an event and is kept goes through it: the event is
+// signed by SNS, but the text in it was written by whichever mail server
+// answered, and it ends up in a table and on a terminal.
+func cleanField(s string, max int) string {
+ s = strings.Map(func(r rune) rune {
+ if r < 32 || r == 127 {
+ return ' '
+ }
+ return r
+ }, strings.TrimSpace(s))
+ if len(s) > max {
+ s = s[:max]
+ for len(s) > 0 && !utf8.ValidString(s) {
+ s = s[:len(s)-1]
+ }
+ }
+ return s
}
// DomainOf extracts the lowercase domain from an address, tolerating a
@@ -67,13 +135,40 @@ func DomainOf(address string) string {
// is worse than no event, because it would be silently dropped from the
// counts that enforce a cap.
func ParseEvent(body []byte) (Event, error) {
+ type recipient struct {
+ EmailAddress string `json:"emailAddress"`
+ Status string `json:"status"`
+ DiagnosticCode string `json:"diagnosticCode"`
+ }
var raw struct {
EventType string `json:"eventType"`
Mail struct {
- Timestamp string `json:"timestamp"`
- MessageID string `json:"messageId"`
- Source string `json:"source"`
+ Timestamp string `json:"timestamp"`
+ MessageID string `json:"messageId"`
+ Source string `json:"source"`
+ Destination []string `json:"destination"`
+ Tags map[string][]string `json:"tags"`
} `json:"mail"`
+ Bounce struct {
+ BounceType string `json:"bounceType"`
+ BounceSubType string `json:"bounceSubType"`
+ BouncedRecipients []recipient `json:"bouncedRecipients"`
+ } `json:"bounce"`
+ Complaint struct {
+ ComplaintFeedbackType string `json:"complaintFeedbackType"`
+ ComplainedRecipients []recipient `json:"complainedRecipients"`
+ } `json:"complaint"`
+ Delivery struct {
+ Recipients []string `json:"recipients"`
+ SMTPResponse string `json:"smtpResponse"`
+ } `json:"delivery"`
+ Reject struct {
+ Reason string `json:"reason"`
+ } `json:"reject"`
+ DeliveryDelay struct {
+ DelayType string `json:"delayType"`
+ DelayedRecipients []recipient `json:"delayedRecipients"`
+ } `json:"deliveryDelay"`
}
if err := json.Unmarshal(body, &raw); err != nil {
return Event{}, fmt.Errorf("mail: parse ses event: %w", err)
@@ -124,5 +219,83 @@ func ParseEvent(body []byte) (Event, error) {
ev.At = at
}
}
+ ev.MessageID = cleanField(ev.MessageID, maxDiagnosticLen)
+ if ids := raw.Mail.Tags["ses:caller-identity"]; len(ids) > 0 {
+ ev.Caller = cleanField(ids[0], maxDiagnosticLen)
+ }
+ for _, to := range raw.Mail.Destination {
+ if to = cleanField(to, maxAddressLen); to != "" {
+ ev.Destination = append(ev.Destination, to)
+ }
+ }
+ outcomes := func(rs []recipient) {
+ for _, r := range rs {
+ to := cleanField(r.EmailAddress, maxAddressLen)
+ if to == "" {
+ continue
+ }
+ ev.Outcomes = append(ev.Outcomes, Outcome{Recipient: to,
+ Status: cleanField(r.Status, maxDetailLen), Diagnostic: cleanField(r.DiagnosticCode, maxDiagnosticLen)})
+ }
+ }
+ // Each type carries its own block; the others are simply empty. Read
+ // only the one this event is, so a malformed message that carries two
+ // cannot have one type's recipients reported under another's name.
+ switch raw.EventType {
+ case EventBounce:
+ ev.Detail.BounceType = cleanField(raw.Bounce.BounceType, maxDetailLen)
+ ev.Detail.BounceSubType = cleanField(raw.Bounce.BounceSubType, maxDetailLen)
+ outcomes(raw.Bounce.BouncedRecipients)
+ case EventComplaint:
+ ev.Detail.ComplaintType = cleanField(raw.Complaint.ComplaintFeedbackType, maxDetailLen)
+ outcomes(raw.Complaint.ComplainedRecipients)
+ case EventDelivery:
+ said := cleanField(raw.Delivery.SMTPResponse, maxDiagnosticLen)
+ for _, to := range raw.Delivery.Recipients {
+ if to = cleanField(to, maxAddressLen); to != "" {
+ ev.Outcomes = append(ev.Outcomes, Outcome{Recipient: to, Diagnostic: said})
+ }
+ }
+ case EventReject:
+ ev.Detail.RejectReason = cleanField(raw.Reject.Reason, maxDiagnosticLen)
+ case EventDeliveryDelay:
+ ev.Detail.DelayType = cleanField(raw.DeliveryDelay.DelayType, maxDetailLen)
+ outcomes(raw.DeliveryDelay.DelayedRecipients)
+ }
return ev, nil
}
+
+// Recipients is who this event is about, one entry per address: the
+// recipients it reports an outcome for, or — for an event about the whole
+// message, a Send or a Reject — everyone it was addressed to. An event that
+// names nobody at all yields a single empty entry, so it is still recorded
+// once rather than not at all.
+func (ev Event) Recipients() []Outcome {
+ if len(ev.Outcomes) > 0 {
+ return ev.Outcomes
+ }
+ if len(ev.Destination) == 0 {
+ return []Outcome{{}}
+ }
+ out := make([]Outcome, 0, len(ev.Destination))
+ for _, to := range ev.Destination {
+ out = append(out, Outcome{Recipient: to})
+ }
+ return out
+}
+
+// Summary is Detail in a few words, for a table: "Permanent/NoEmail",
+// "abuse", "Bad content". Empty when the event has nothing to add.
+func (d EventDetail) Summary() string {
+ switch {
+ case d.BounceType != "" && d.BounceSubType != "":
+ return d.BounceType + "/" + d.BounceSubType
+ case d.BounceType != "":
+ return d.BounceType
+ case d.ComplaintType != "":
+ return d.ComplaintType
+ case d.RejectReason != "":
+ return d.RejectReason
+ }
+ return d.DelayType
+}
diff --git a/internal/mail/events_test.go b/internal/mail/events_test.go
index f23326d3c..89853372a 100644
--- a/internal/mail/events_test.go
+++ b/internal/mail/events_test.go
@@ -1,6 +1,9 @@
package mail
-import "testing"
+import (
+ "strings"
+ "testing"
+)
// SES's event JSON nests the From address under mail.source and the type
// under eventType. Attribution is by SENDING DOMAIN, because that is the
@@ -161,3 +164,154 @@ func TestParseEventIgnoresMalformedTimestamp(t *testing.T) {
t.Fatalf("At = %v, want zero (timestamp ignored)", ev.At)
}
}
+
+// What SES publishes for a bounce, in the shape its event destinations
+// send: the message's own envelope under mail, the automatic tags naming
+// the sending identity, and a bounce block that says, per recipient, what
+// the receiving server answered.
+const bounceEvent = `{
+ "eventType": "Bounce",
+ "bounce": {
+ "bounceType": "Permanent",
+ "bounceSubType": "NoEmail",
+ "bouncedRecipients": [
+ {"emailAddress": "ann@example.com", "action": "failed", "status": "5.1.1",
+ "diagnosticCode": "smtp; 550 5.1.1 <ann@example.com>: Recipient address rejected"}
+ ],
+ "timestamp": "2026-09-15T08:00:02.000Z",
+ "feedbackId": "0102019-feedback"
+ },
+ "mail": {
+ "timestamp": "2026-09-15T08:00:00.000Z",
+ "source": "bookings@lgc.li",
+ "messageId": "0102019abc-7c8d-000000",
+ "destination": ["ann@example.com"],
+ "tags": {
+ "ses:caller-identity": ["carlos-mail-beb-jb-book"],
+ "ses:from-domain": ["lgc.li"]
+ }
+ }
+}`
+
+func TestParseEventReadsWhoBouncedAndWhy(t *testing.T) {
+ ev, err := ParseEvent([]byte(bounceEvent))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if ev.MessageID != "0102019abc-7c8d-000000" || ev.Caller != "carlos-mail-beb-jb-book" {
+ t.Errorf("message %q, caller %q", ev.MessageID, ev.Caller)
+ }
+ if len(ev.Outcomes) != 1 {
+ t.Fatalf("outcomes = %+v, want one", ev.Outcomes)
+ }
+ o := ev.Outcomes[0]
+ if o.Recipient != "ann@example.com" || o.Status != "5.1.1" || !strings.Contains(o.Diagnostic, "Recipient address rejected") {
+ t.Errorf("outcome = %+v", o)
+ }
+ if got := ev.Detail.Summary(); got != "Permanent/NoEmail" {
+ t.Errorf("summary = %q", got)
+ }
+}
+
+func TestParseEventReadsEachTypesOwnRecipients(t *testing.T) {
+ for _, c := range []struct {
+ name, body, recipient, diagnostic, summary string
+ }{
+ {"delivery", `{"eventType":"Delivery","mail":{"source":"a@lgc.li","messageId":"m1","destination":["bob@example.com"]},
+ "delivery":{"recipients":["bob@example.com"],"smtpResponse":"250 2.0.0 OK 1726387202"}}`,
+ "bob@example.com", "250 2.0.0 OK 1726387202", ""},
+ {"complaint", `{"eventType":"Complaint","mail":{"source":"a@lgc.li","messageId":"m2"},
+ "complaint":{"complaintFeedbackType":"abuse","complainedRecipients":[{"emailAddress":"cat@example.com"}]}}`,
+ "cat@example.com", "", "abuse"},
+ {"delay", `{"eventType":"DeliveryDelay","mail":{"source":"a@lgc.li","messageId":"m3"},
+ "deliveryDelay":{"delayType":"MailboxFull","delayedRecipients":[{"emailAddress":"dan@example.com","status":"4.2.2","diagnosticCode":"452 mailbox full"}]}}`,
+ "dan@example.com", "452 mailbox full", "MailboxFull"},
+ } {
+ t.Run(c.name, func(t *testing.T) {
+ ev, err := ParseEvent([]byte(c.body))
+ if err != nil {
+ t.Fatal(err)
+ }
+ rs := ev.Recipients()
+ if len(rs) != 1 || rs[0].Recipient != c.recipient || rs[0].Diagnostic != c.diagnostic {
+ t.Errorf("recipients = %+v", rs)
+ }
+ if got := ev.Detail.Summary(); got != c.summary {
+ t.Errorf("summary = %q, want %q", got, c.summary)
+ }
+ })
+ }
+}
+
+// A Send or a Reject is about the whole message, so it concerns everyone
+// the message was addressed to. An event that names nobody is still one
+// event, not none.
+func TestAnEventAboutTheWholeMessageConcernsEveryAddressee(t *testing.T) {
+ ev, err := ParseEvent([]byte(`{"eventType":"Reject","mail":{"source":"a@lgc.li","messageId":"m4",
+ "destination":["one@example.com","two@example.com"]},"reject":{"reason":"Bad content"}}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if rs := ev.Recipients(); len(rs) != 2 || rs[1].Recipient != "two@example.com" {
+ t.Errorf("recipients = %+v", rs)
+ }
+ if ev.Detail.Summary() != "Bad content" {
+ t.Errorf("summary = %q", ev.Detail.Summary())
+ }
+ bare, _ := ParseEvent([]byte(`{"eventType":"Send","mail":{"source":"a@lgc.li"}}`))
+ if rs := bare.Recipients(); len(rs) != 1 || rs[0].Recipient != "" {
+ t.Errorf("an event naming nobody should still be one entry: %+v", rs)
+ }
+}
+
+// A type's recipients are read from its own block only. A malformed
+// message carrying a delivery block under a bounce must not report the
+// delivered address as bounced.
+func TestParseEventReadsOnlyTheBlockItsTypeNames(t *testing.T) {
+ ev, err := ParseEvent([]byte(`{"eventType":"Bounce","mail":{"source":"a@lgc.li"},
+ "delivery":{"recipients":["bob@example.com"]}}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(ev.Outcomes) != 0 {
+ t.Errorf("a bounce reported the delivery block's recipients: %+v", ev.Outcomes)
+ }
+}
+
+// The text in an event was written by whichever server answered. It is
+// kept, so it is kept clean: no control characters, and bounded.
+func TestEventTextIsCleanedBeforeItIsKept(t *testing.T) {
+ long := strings.Repeat("x", 5000)
+ ev, err := ParseEvent([]byte(`{"eventType":"Bounce","mail":{"source":"a@lgc.li"},"bounce":{"bounceType":"Permanent",
+ "bouncedRecipients":[{"emailAddress":"ann@example.com\r\nX-Injected: yes","diagnosticCode":"` + long + `"}]}}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ o := ev.Outcomes[0]
+ if strings.ContainsAny(o.Recipient, "\r\n") {
+ t.Errorf("control characters survived: %q", o.Recipient)
+ }
+ if len(o.Diagnostic) > maxDiagnosticLen {
+ t.Errorf("diagnostic kept at %d bytes", len(o.Diagnostic))
+ }
+}
+
+func TestAppOfCallerReadsBackTheDeliveredIdentity(t *testing.T) {
+ for _, c := range []struct {
+ caller, acct, app string
+ ok bool
+ }{
+ {UserName("beb", "jb-book"), "beb", "jb-book", true},
+ {StandaloneUserName("beb", "jb-book", "laptop"), "beb", "jb-book", true},
+ {"arn:aws:iam::123456789012:user/carlos/mail/carlos-mail-beb-jb-book", "beb", "jb-book", true},
+ {"carlos-mail-beb", "", "", false}, // no app
+ {"carlos-mail-BEB-jb-book", "", "", false}, // not an account sqid
+ {"someone-else-entirely", "", "", false}, // not ours
+ {"", "", "", false},
+ } {
+ acct, app, ok := AppOfCaller(c.caller)
+ if acct != c.acct || app != c.app || ok != c.ok {
+ t.Errorf("AppOfCaller(%q) = %q, %q, %v; want %q, %q, %v", c.caller, acct, app, ok, c.acct, c.app, c.ok)
+ }
+ }
+}
diff --git a/internal/mail/policy.go b/internal/mail/policy.go
index 82df3eacc..100db4d8a 100644
--- a/internal/mail/policy.go
+++ b/internal/mail/policy.go
@@ -9,6 +9,8 @@ import (
"encoding/json"
"fmt"
"strings"
+
+ "github.com/carlosframework/platform/internal/release"
)
// PolicyName is the single inline policy per user. One name, one document —
@@ -24,6 +26,37 @@ const UserPath = "/carlos-mail/"
// UserName is the principal that owns an app's DELIVERED credentials.
func UserName(acct, app string) string { return "carlos-mail-" + acct + "-" + app }
+// AppOfCaller reads the account and app back out of the IAM identity SES
+// reports as a message's sender (Event.Caller) — the inverse of UserName,
+// and of StandaloneUserName, whose credential sends for the same app. It is
+// unambiguous because an account sqid is lowercase letters only
+// (release.ValidAccount), so the first hyphen after the prefix is where the
+// account ends, and everything after it is the app, hyphens and all.
+//
+// SES is documented to report the identity's name; an ARN is accepted too,
+// by its last path segment, so a change in which of the two SES sends does
+// not quietly stop naming the sender. Anything that is not one of CARLOS's
+// own identities answers false, and the event falls back to being
+// attributed by its sending domain alone.
+func AppOfCaller(caller string) (acct, app string, ok bool) {
+ name := caller
+ if i := strings.LastIndex(name, "/"); i >= 0 {
+ name = name[i+1:]
+ }
+ rest, found := strings.CutPrefix(name, "carlos-mail-")
+ if !found {
+ return "", "", false
+ }
+ if i := strings.Index(rest, "+s+"); i >= 0 {
+ rest = rest[:i]
+ }
+ acct, app, found = strings.Cut(rest, "-")
+ if !found || !release.ValidAccount(acct) || !release.ValidApp(app) {
+ return "", "", false
+ }
+ return acct, app, true
+}
+
// TenantName is the SES tenant CARLOS creates for a CARLOS account.
//
// It is prefixed, never the bare sqid, because the SES tenant namespace is
diff --git a/mail_cli.go b/mail_cli.go
index 1871adf4d..1a96f201f 100644
--- a/mail_cli.go
+++ b/mail_cli.go
@@ -48,8 +48,10 @@ import (
"net"
"net/http"
"net/smtp"
+ "net/url"
"os"
"sort"
+ "strconv"
"strings"
"text/tabwriter"
"time"
@@ -76,6 +78,12 @@ const emailUsage = `usage: carlos email <enable|domains|status|test|credentials|
status --app <a> domains and their per-region verification state,
today's counts against the cap, and the pause
state.
+ events --app <a> [--since <dur|time>] [--message <id>] [--to <addr>] [-n <count>] [--json]
+ what happened to each message the app sent —
+ delivered, bounced, complained about, delayed —
+ newest first, with what the receiving server
+ said. Kept for 30 days. --message takes the id SES
+ gave the message when it was sent.
test --app <a> --to <addr>
send a REAL message and report the result — the
verify half of the no-infra rule: this is the
@@ -126,6 +134,8 @@ func runEmail(args []string) {
runEmailDomains(args[1:])
case "status":
runEmailStatus(args[1:])
+ case "events":
+ runEmailEvents(args[1:])
case "test":
runEmailTest(args[1:])
case "credentials":
@@ -804,6 +814,173 @@ func runEmailStatus(args []string) {
printEmailStatus(stdout, res)
}
+// apiEmailEvent is one row of `carlos email events`, as the console sends it.
+type apiEmailEvent struct {
+ MessageID string `json:"message_id"`
+ Type string `json:"type"`
+ Recipient string `json:"recipient"`
+ Status string `json:"status,omitempty"`
+ Diagnostic string `json:"diagnostic,omitempty"`
+ Detail string `json:"detail,omitempty"`
+ BySender bool `json:"by_sender"`
+ At time.Time `json:"at"`
+}
+
+type apiEmailEventsReply struct {
+ Events []apiEmailEvent `json:"events"`
+ RetentionDays int `json:"retention_days"`
+}
+
+func (s *apiSession) emailEvents(ctx context.Context, a apiAcct, app string, q url.Values) (apiEmailEventsReply, error) {
+ tail := "/email/events"
+ if len(q) > 0 {
+ tail += "?" + q.Encode()
+ }
+ status, body, err := s.getJSON(ctx, apiPath(a, app, tail))
+ if err != nil {
+ return apiEmailEventsReply{}, err
+ }
+ if status != http.StatusOK {
+ return apiEmailEventsReply{}, mailAPIError(status, body)
+ }
+ var out apiEmailEventsReply
+ if err := json.Unmarshal(body, &out); err != nil {
+ return apiEmailEventsReply{}, fmt.Errorf("email events: unreadable response: %w", err)
+ }
+ return out, nil
+}
+
+// sinceParam turns --since into the RFC 3339 time the console reads: a
+// duration counts back from now, anything else must already be a time.
+// Empty stays empty — the whole retention window.
+func sinceParam(raw string, now time.Time) (string, error) {
+ raw = strings.TrimSpace(raw)
+ if raw == "" {
+ return "", nil
+ }
+ if d, err := time.ParseDuration(raw); err == nil {
+ if d < 0 {
+ d = -d
+ }
+ return now.Add(-d).UTC().Format(time.RFC3339), nil
+ }
+ if t, err := time.Parse(time.RFC3339, raw); err == nil {
+ return t.UTC().Format(time.RFC3339), nil
+ }
+ return "", fmt.Errorf("--since %q is neither a duration (24h) nor a time (2026-09-15T08:00:00Z)", raw)
+}
+
+func runEmailEvents(args []string) {
+ fs := flag.NewFlagSet("email events", flag.ExitOnError)
+ app := appFlag(fs, "app whose email events to show")
+ acctRaw, acctIDRaw, _ := accountFlags(fs)
+ since := fs.String("since", "", "only events since: a duration back from now (24h), or an RFC 3339 time (default: all 30 days kept)")
+ message := fs.String("message", "", "only this message: the id SES gave it when it was sent")
+ to := fs.String("to", "", "only events about this recipient address")
+ n := fs.Int("n", 0, "at most this many events (default 100, max 1000)")
+ asJSON := fs.Bool("json", false, "print the events as JSON, one array, for a script")
+ consoleFlag := fs.String("console", "", "console origin (API mode only; default: the only logged-in one)")
+ fs.Usage = flagUsage(fs, "")
+ fs.Parse(args)
+ name := strings.TrimSpace(*app)
+ usage := "usage: carlos email events --app <a> [--since <dur|time>] [--message <id>] [--to <addr>] [-n <count>] [--json] [--account <sqid-or-name>] [--console <origin>]"
+ if name != "" {
+ if err := validAppName(name); err != nil {
+ log.Fatalf("email events: %v", err)
+ }
+ }
+ q := url.Values{}
+ sinceAt, err := sinceParam(*since, time.Now())
+ if err != nil {
+ log.Fatalf("email events: %v", err)
+ }
+ if sinceAt != "" {
+ q.Set("since", sinceAt)
+ }
+ if v := strings.TrimSpace(*message); v != "" {
+ q.Set("message", v)
+ }
+ if v := strings.TrimSpace(*to); v != "" {
+ q.Set("to", v)
+ }
+ if *n > 0 {
+ q.Set("limit", strconv.Itoa(*n))
+ }
+
+ sess := requireMailAPISession(*consoleFlag)
+ if !*asJSON {
+ printUsing(stdout, sess)
+ }
+ acct := resolvedAccount(sess, acctRaw, acctIDRaw)
+ name = resolvedApp(sess, acct, name, usage)
+ res, err := sess.emailEvents(context.Background(), acct, name, q)
+ if err != nil {
+ log.Fatalf("email events: %v", err)
+ }
+ if *asJSON {
+ if res.Events == nil {
+ res.Events = []apiEmailEvent{}
+ }
+ enc := json.NewEncoder(stdout)
+ enc.SetIndent("", " ")
+ enc.Encode(res.Events)
+ return
+ }
+ printEmailEvents(stdout, name, res)
+}
+
+// printEmailEvents is one line per recipient per event, newest first: when,
+// what happened, to whom, and the message it happened to — then, indented
+// beneath, what the receiving server actually said, which is the part that
+// tells a member what to do about a bounce.
+func printEmailEvents(w io.Writer, app string, res apiEmailEventsReply) {
+ days := res.RetentionDays
+ if days == 0 {
+ days = 30
+ }
+ if len(res.Events) == 0 {
+ mutedf(w, "no email events for %s in the last %d days that match — events arrive a few seconds after SES hears back from the receiving server", app, days)
+ return
+ }
+ width := 0
+ for _, e := range res.Events {
+ if n := len(eventWord(e)); n > width {
+ width = n
+ }
+ }
+ for _, e := range res.Events {
+ to := e.Recipient
+ if to == "" {
+ to = "(no recipient named)"
+ }
+ fmt.Fprintf(w, "%s %-*s %s %s\n", e.At.Local().Format("2006-01-02 15:04:05"), width, eventWord(e), to, e.MessageID)
+ said := e.Diagnostic
+ if e.Status != "" && !strings.Contains(said, e.Status) {
+ said = strings.TrimSpace(e.Status + " " + said)
+ }
+ if said != "" {
+ mutedf(w, " %s", said)
+ }
+ }
+ mutedf(w, "%d event(s), newest first — kept for %d days", len(res.Events), days)
+}
+
+// eventWord is the event type as a member reads it, with the kind of bounce
+// or complaint where SES said: "bounce (Permanent/NoEmail)".
+func eventWord(e apiEmailEvent) string {
+ word := map[string]string{
+ "Send": "sent", "Delivery": "delivered", "Bounce": "bounced", "Complaint": "complaint",
+ "Reject": "rejected", "DeliveryDelay": "delayed",
+ }[e.Type]
+ if word == "" {
+ word = strings.ToLower(e.Type)
+ }
+ if e.Detail != "" {
+ word += " (" + e.Detail + ")"
+ }
+ return word
+}
+
// printEmailStatus is the one view a member with no box access and no AWS
// console has of every domain's per-region verification state, today's
// counts against the cap, and the pause state.
diff --git a/mail_cli_test.go b/mail_cli_test.go
index 39fe68999..8bb049461 100644
--- a/mail_cli_test.go
+++ b/mail_cli_test.go
@@ -1108,3 +1108,57 @@ func TestPrintEmailStatusDoesNotWarnOnAnUnverifiedDomain(t *testing.T) {
t.Fatalf("status output warns about events on a still-verifying domain:\n%s", out)
}
}
+
+// `carlos email events` puts what happened first and what the receiving
+// server said beneath it — the second line is what tells a member whether
+// a bounce was a typo in the address or a full mailbox.
+func TestPrintEmailEventsSaysWhatHappenedAndWhy(t *testing.T) {
+ var buf bytes.Buffer
+ at := time.Date(2026, 9, 15, 8, 0, 2, 0, time.UTC)
+ printEmailEvents(&buf, "jb-book", apiEmailEventsReply{RetentionDays: 30, Events: []apiEmailEvent{
+ {MessageID: "0102-ann", Type: "Bounce", Recipient: "ann@example.com", Status: "5.1.1",
+ Diagnostic: "smtp; 550 5.1.1 Recipient address rejected", Detail: "Permanent/NoEmail", At: at},
+ {MessageID: "0102-bob", Type: "Delivery", Recipient: "bob@example.com", Diagnostic: "250 2.0.0 OK", At: at.Add(-time.Minute)},
+ }})
+ out := buf.String()
+ for _, want := range []string{
+ "bounced (Permanent/NoEmail)", "ann@example.com", "0102-ann", "Recipient address rejected",
+ "delivered", "bob@example.com", "250 2.0.0 OK",
+ "2 event(s)", "30 days",
+ } {
+ if !strings.Contains(out, want) {
+ t.Errorf("output is missing %q:\n%s", want, out)
+ }
+ }
+ if strings.Index(out, "0102-ann") > strings.Index(out, "0102-bob") {
+ t.Errorf("events should print in the order the console sent them, newest first:\n%s", out)
+ }
+ // The status code is already in the diagnostic here; it is not said twice.
+ if strings.Count(out, "5.1.1") != 1 {
+ t.Errorf("the status code was repeated:\n%s", out)
+ }
+
+ buf.Reset()
+ printEmailEvents(&buf, "jb-book", apiEmailEventsReply{RetentionDays: 30})
+ if !strings.Contains(buf.String(), "no email events for jb-book") {
+ t.Errorf("an empty log should say so, and why it might be empty:\n%s", buf.String())
+ }
+}
+
+func TestSinceParamTakesADurationOrATime(t *testing.T) {
+ now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
+ for _, c := range []struct{ in, want string }{
+ {"", ""},
+ {"24h", "2026-09-14T12:00:00Z"},
+ {"90m", "2026-09-15T10:30:00Z"},
+ {"2026-09-10T08:00:00+01:00", "2026-09-10T07:00:00Z"},
+ } {
+ got, err := sinceParam(c.in, now)
+ if err != nil || got != c.want {
+ t.Errorf("sinceParam(%q) = %q, %v; want %q", c.in, got, err, c.want)
+ }
+ }
+ if _, err := sinceParam("yesterday", now); err == nil {
+ t.Error("a word that is neither should be refused, not sent to the console")
+ }
+}