🤖 Moderation under E2EE — the suite, its trust model, and its honest limits

Status: shipped (issue #578, the mod-* PR stack, 2026-07-12). This documents what the moderation suite does, what the server learns for each piece, how it interacts with access levels, and the two E2EE realities the design is built around — including the franking decision at the end.

The one-line frame: moderation here is report-driven by design. Admins hold no skeleton key — they see only the conversations they're in, exactly like any member. What they moderate is what members surface, plus content-blind signals (rates, volumes, repeat counts) the server can compute without reading anything.

Member tools

| tool | what it does | where it lives | |---|---|---| | Mute a chat | No notifications, no unread dot; messages keep arriving quietly. | Thread ⋮ menu (web + iOS + Mac) | | Mute a person | Their messages fold to a "Muted message" stub (tap to read); they never notify. Private — they're not told. | Profile card, message long-press | | DM privacy | "Who can message you directly": everyone in the space, or no one. Enforced server-side at DM creation. | Edit profile (web), Settings (native) | | End conversation | Closes a DM for both parties; the server refuses new messages/edits/typing until someone reopens it. | DM ⋮ menu | | Report | Message or person → category + note + quoted evidence, sealed to each admin's key. | Message long-press, profile card | | Block | Mute + end DM + server-enforced no new contact: no fresh DMs either way, no group adds either way, no pushes from them. | Profile card, report sheet |

Admin tools

| tool | what it does | |---|---| | Reports inbox | The Admin app's queue: decrypted report content (sealed to each admin), repeat signals (how often the reporter files / the reported is named), invite lineage ("invited by X"). | | Warn | A formal notice, sent as an ordinary E2EE DM with a warn flag inside the ciphertext — clients render the "notice from an admin" framing; the server records only that the rung was used. | | Remove from group / access level / space | The existing removal levers, wired as one-tap resolutions off a report. Removing an access level is the demote rung (see tiers, below). | | Remove & ban | Removal plus a persistent denial keyed on the member's public key: that key can't re-register through any invite — admin, member, or cohort. | | Audit log | Append-only record of every moderation action: who, what, whom, when, off which report. Includes invite-pause flips. | | Invite pause | The existing "Members can invite people" switch is the panic lever during a spam wave; flips are audited. | | Spam brakes | Content-blind token buckets: messages (burst 30, ~30/min), new DMs (~10/h; day-one members ~3/h), invite minting (~20/day). Admins exempt. |

What the server learns, per piece

The E2EE rule holds throughout: every server-side moderation record is connection/action metadata — never content, never keys.

• Mute a chat — a per-user thread flag (muted_threads), the same class as pinned/hidden. Deliberately server-visible so the server can skip pushes entirely (an iOS NSE can't fully drop a delivered push without Apple's filtering entitlement; not sending is strictly better).
• Mute a person — nothing. The muted set lives in a blob sealed to the member's own key (users.prefs_blob); the server syncs ciphertext it cannot read. Notification suppression happens after decryption, on-device.
• DM privacy — one enum on the user row.
• Close DM / block / ban — connection records (closed_dms, blocks, bans): who cut contact with whom, never why. The blocked/banned party gets the same generic refusals a closed DM door gives.
• Report — queue metadata (reporter, reported, thread id, status, outcome) plus per-admin sealed blobs the server can't open. The category and every word travel inside the blobs.
• Warn — an audit row saying a warning was issued. The warning's words are an ordinary E2EE DM.
• Audit log — plaintext action metadata, admin-served only. The same class as the roster.

The tiers (access levels) tie-ins

• Demote is a rung on the ladder. A report can be resolved by removing one of the reported member's access levels (unassignTier) — losing silver without losing the space. Audited as level.removed.
• Bans cover cohort invites. A cohort invite assigns a level at claim, but a claim only becomes a member when a public key registers — and that is where the ban check lives. One gate, every invite path.
• Roles are not tiers, still. Moderation powers ride users.role (admin), never key material. role='moderator' remains reserved and unenforced — wiring moderators into the queue (they'd need report blobs sealed to them too) is deliberate future work, noted below.

The two hard E2EE realities (and what we did about them)

Reports are testimony, not proof

A report from an encrypted chat is a claim: the server holds only ciphertext, so admins cannot cryptographically check that "they said X". The suite ships the honest v1:

• The report sheet says so: "Reports can't be verified automatically — admins read them and use their judgment." The queue repeats it.
• The queue shows pattern signals instead of pretending at proof: how often this reporter files, how often this member is named, who invited them.
• Report abuse is a moderation problem of its own: filings are capped (5/hour, content-blind), reporter history is visible, and nothing ever auto-punishes on report count.

Admins can't see who anyone is server-side — or read any thread they're not in

Names and photos are E2EE, so the token-auth admin console sees an opaque roster and only queue metadata. Everything that needs words or names — the report content, the audit log's names, the warn composer — lives in the Admin app, where the admin is a member holding keys. The console keeps its key-blind operations only.

Honest limits (say them out loud)

• A ban stops the key, not the person. A determined person with a fresh invite and a fresh keypair is a new stranger to the server. That's why the ban button sits next to invite lineage and the invite pause — the actual countermeasures for re-entry.
• Removal doesn't rekey. A removed/banned member who exfiltrated keys client-side plus a later server compromise could decrypt future traffic in their old threads. Re-key-on-removal is #541 Q10 — a real protocol change, tracked there, not silently promised here.
• Mute is one-sided by design. "You stop seeing them", never "they stop existing". The muted party can still read the group and see your profile — within one space, "can't see I exist" isn't enforceable and we don't pretend otherwise.
• Blocking hides nothing in shared groups. It cuts contact (DMs, adds, notifications) and folds their messages away for you; you still share the room.

The franking decision (envelope room: not needed)

Message franking (the Messenger construction) would make reports cryptographically verifiable: at send time the sender's client commits to the plaintext (an HMAC whose key is revealed on report), the server blindly countersigns the commitment at delivery, and on report the admins check the revealed plaintext against the commitment the server saw. Fabricating "they said X" then becomes detectable without the server ever reading anything.

Decision: build nothing now, reserve nothing now — because nothing needs reserving. The issue asked us to decide early whether to hold room in the payload envelope. We looked at the wire format and the answer is that both places franking would touch are already extensible:

1. The encrypted payload is JSON. A future sender adds frank: {c, …} (the commitment material) as one more optional field — exactly how warn, replyTo, and card were added. Old clients ignore unknown fields by construction; the event-vector suite is where the field gets pinned when it becomes real.
2. The server frame is JSON too. The countersignature (server MAC over the commitment + timestamp) would ride the stored message row / WS frame (wsOut), also additive.

So the only genuinely new machinery is behavioural — the server keeping a countersigning key and stamping every accepted message, clients verifying on report — and none of it is made cheaper by reserving bytes today. What we have done now is keep the report content shape versioned ({"v":1, …}) so a v:2 report carrying franking material is a clean upgrade.

Revisit franking when (a) spaces are big enough that fabricated reports are an observed problem, not a theoretical one, and (b) #541's rekey questions are settled — the two share the "protocol change with client fan-out" cost profile and should be scheduled together.

Deliberate scoping

• The admin queue is web-only (the Admin app). The native apps ship every member tool; they deliberately track no admin surfaces yet — the same scoping as native access-level management. Native members still see warn framing, report sheets, mute/block — everything a non-admin touches.
• Per-space only. Cross-instance moderation (shared ban lists, platform obligations) is #574's territory.
• role='moderator' stays unenforced. Wiring it up means: sealing report blobs to moderators too, letting them warn/remove-from-thread but not remove-from-space/ban, and deciding whether members can see who moderates. Its own issue when wanted.
• Brake ceilings are constants (ratelimit.go), far above human pace. Make them per-space config only if a real space actually needs different ones.