amadan
agent: bots only reply when actually addressed (bare-name classifier gate) (#70)
A bare "Claude" anywhere in a group message used to trigger a reply, so talking *about* a bot summoned it. Group triggers are now layered: - @Name -> always replies (deterministic, no extra API call) - bare name -> a cheap Haiku yes/no call decides whether the bot is being spoken to or merely mentioned; errors/ambiguity default to silence - DMs unchanged (always reply) Classifier model is claude-haiku-4-5, overridable via LCHAT_CLASSIFIER_MODEL. Docs, the SAFETY preamble, and Eleven's app guide updated to match; changelog entry added for the Announcer.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
6 files changed,
+66
−12
agent/README.md+6 −3agent/config.mjs+3 −0agent/eleven.mjs+1 −1agent/runtime.mjs+49 −7changelog/2026-07-02-bots-reply-when-spoken-to.md+4 −0docs/bots.md+3 −1
diff --git a/agent/README.md b/agent/README.md| index 868ceca..7f16b5a 100644 |
| --- a/agent/README.md |
| +++ b/agent/README.md |
| @@ -8,7 +8,7 @@ its wire format is guaranteed to match the app. |
| | agent | what it does | |
| |---|---| |
| -| **Claude** | The plain assistant: replies to every DM, and on @mention in groups. (`runtime.mjs`, identity via `LCHAT_SESSION`/`LCHAT_INVITE`.) | |
| +| **Claude** | The plain assistant: replies to every DM; in groups an @mention always gets a reply, and a bare-name mention only does if a cheap classifier judges the bot is actually being addressed (not just talked about). (`runtime.mjs`, identity via `LCHAT_SESSION`/`LCHAT_INVITE`.) | |
| | **Eleven** | The app guide: same chat loop with a shipped-in persona + app reference (updated with every deploy), and it **DMs each new member a welcome** when they join. (`eleven.mjs`, `LCHAT_ELEVEN_*`.) | |
| | **Announcer** | Write-only deploy notes: polls the embedded changelog (`/api/changelog`) and posts what's new to groups it's added to. Never reads messages, never calls the Claude API. (`announcer.mjs`, `LCHAT_ANNOUNCER_*`.) | |
| @@ -20,7 +20,7 @@ console, its `#fragment` passed once as the matching `*_INVITE` env var. |
| | | | |
| |---|---| |
| -| `ANTHROPIC_API_KEY` | The only recurring secret + cost. Every DM and every group @mention is one Claude API call billed to your account. | |
| +| `ANTHROPIC_API_KEY` | The only recurring secret + cost. Every DM and every group @mention is one Claude API call billed to your account (bare-name mentions also spend a fraction-of-a-cent Haiku call to decide whether to answer — `LCHAT_CLASSIFIER_MODEL` overrides). | |
| | `LCHAT_BASE` | Your instance URL, e.g. `https://test.luan.chat`. | |
| | `LCHAT_INVITE` | **One-time.** The `#fragment` of a `/join/…` link minted in the admin console (tick "This is an agent"). Spent after provisioning. | |
| @@ -44,7 +44,10 @@ LCHAT_BASE=https://test.luan.chat npm start |
| ``` |
| Then, in the app, add the agent to a group or start a DM with it. It replies to |
| -every message in a DM, and only when @mentioned by name in a group. |
| +every message in a DM. In a group, an explicit @mention is a guaranteed reply; |
| +a message that just uses its name gets a reply only when a small classifier |
| +call agrees the bot is being spoken *to* rather than *about* — so chatting |
| +about a bot doesn't summon it. |
| ## Eleven & the Announcer |
diff --git a/agent/config.mjs b/agent/config.mjs| index db65464..3c43a03 100644 |
| --- a/agent/config.mjs |
| +++ b/agent/config.mjs |
| @@ -13,6 +13,9 @@ export const WS_BASE = BASE.replace(/^http/, "ws"); |
| export const AGENT_NAME = process.env.LCHAT_AGENT_NAME || "Claude"; |
| export const MODEL = process.env.LCHAT_MODEL || "claude-sonnet-5"; |
| +// The bare-name "am I being addressed?" gate in runtime.mjs — a tiny yes/no |
| +// call, so it always uses the cheapest model regardless of the bot's MODEL. |
| +export const CLASSIFIER_MODEL = process.env.LCHAT_CLASSIFIER_MODEL || "claude-haiku-4-5"; |
| // Session lives next to this module by default so `node agent.mjs` works from |
| // any cwd; override with LCHAT_SESSION. |
diff --git a/agent/eleven.mjs b/agent/eleven.mjs| index 7cab364..f3cbb0c 100644 |
| --- a/agent/eleven.mjs |
| +++ b/agent/eleven.mjs |
| @@ -42,7 +42,7 @@ export const ELEVEN_PERSONA = |
| export const ELEVEN_DOCS = [ |
| "lchat basics: every chat is end-to-end encrypted — the server (and its operator) can never read messages.", |
| "Start a chat: tap ➕, pick a person for a DM, or 'New group' to pick several people and name the group.", |
| - "Helpers: bots live under the Helpers section of the ➕ picker; each says what it does. Mention a bot by name (@Claude) to get its attention in a group; in a DM it always replies.", |
| + "Helpers: bots live under the Helpers section of the ➕ picker; each says what it does. @mention a bot (@Claude) in a group for a guaranteed reply — speaking to it by name also works, but it stays quiet when a message merely talks about it. In a DM it always replies.", |
| "Invite someone new: if the admin enabled member invites, ➕ → 'Invite someone' creates a private link — share it only with that person, since the link can read the chat it opens.", |
| "Groups: tap the group name for members; add people via the menu; swipe right on a message (or use the message menu) to reply; double-tap (touch) or press-and-hold (desktop) to react.", |
| "Photos & files: the paperclip attaches them; images are shrunk and stripped of location data before upload, and everything is encrypted.", |
diff --git a/agent/runtime.mjs b/agent/runtime.mjs| index 1416f65..1a2109a 100644 |
| --- a/agent/runtime.mjs |
| +++ b/agent/runtime.mjs |
| @@ -8,7 +8,7 @@ import fs from "node:fs"; |
| import Anthropic from "@anthropic-ai/sdk"; |
| import WebSocket from "ws"; |
| import { importKey, encrypt, decrypt, unwrapThreadKey } from "../web/static/crypto.js"; |
| -import { BASE, WS_BASE, MODEL } from "./config.mjs"; |
| +import { BASE, WS_BASE, MODEL, CLASSIFIER_MODEL } from "./config.mjs"; |
| const log = (m) => { |
| try { |
| @@ -25,7 +25,7 @@ const SAFETY = (name) => |
| `You are ${name}, an AI assistant and a member of lchat — a small, private, ` + |
| `end-to-end encrypted group chat; the server and its operator can't read messages, ` + |
| `and that privacy is the whole point of the app. You are one participant among the people here. ` + |
| - `You may be in a one-to-one DM or in a group; in a group you're brought in when someone mentions you by name. ` + |
| + `You may be in a one-to-one DM or in a group; in a group you're brought in only when someone @mentions you or speaks to you directly by name. ` + |
| `You can only take part in the conversation — no reminders, notifications, files, or web access — ` + |
| `so never promise to do those things; if asked, say you can't yet. ` + |
| `This chat may include children: always be kind and age-appropriate, and never say anything scary, adult, or unsafe. ` + |
| @@ -107,10 +107,50 @@ export function runBot({ name, session, builtinPersona = "", builtinDocs = "" }) |
| } |
| } |
| - // DMs to the bot: always answer. Groups: only when @mentioned by name. |
| - function wantsReply(kind, text) { |
| - if (kind === "dm") return true; |
| - return new RegExp(`(^|[^a-z0-9])@?${name}\\b`, "i").test(text || ""); |
| + // DMs to the bot: always answer. Groups are layered: an explicit @Name is a |
| + // guaranteed, deterministic reply; a bare name is ambiguous — people talk |
| + // *about* a bot as often as *to* it — so it only earns a reply if the |
| + // classifier below agrees we're being addressed. Anything else is ignored. |
| + function replyTrigger(kind, text) { |
| + if (kind === "dm") return "always"; |
| + if (new RegExp(`(^|[^a-z0-9])@${name}\\b`, "i").test(text || "")) return "always"; |
| + if (new RegExp(`(^|[^a-z0-9])${name}\\b`, "i").test(text || "")) return "classify"; |
| + return null; |
| + } |
| + |
| + // The bare-name gate: one tiny yes/no call over the last few messages. Errs |
| + // toward silence (failure or ambiguity = no reply) — @Name is the guaranteed |
| + // path, so a false negative here is always recoverable by the sender. |
| + async function isAddressed(threadId, key) { |
| + const { messages = [] } = await get(`/api/messages?thread=${encodeURIComponent(threadId)}&limit=6`); |
| + const lines = []; |
| + for (const m of messages) { |
| + try { |
| + const p = await decrypt(key, m.blob); |
| + if (p.text) lines.push(`${p.name}: ${p.text}`); |
| + } catch {} |
| + } |
| + if (!lines.length) return false; |
| + try { |
| + const r = await api().messages.create({ |
| + model: CLASSIFIER_MODEL, |
| + max_tokens: 5, |
| + system: |
| + `"${name}" is a chat bot in a group conversation. Given the recent messages, decide whether ` + |
| + `the LAST message is addressed to ${name} and expects ${name} to reply — as opposed to merely ` + |
| + `mentioning ${name} or talking about ${name}. Answer with exactly "yes" or "no".`, |
| + messages: [{ role: "user", content: lines.join("\n") }], |
| + }); |
| + return /yes/i.test( |
| + r.content |
| + .filter((b) => b.type === "text") |
| + .map((b) => b.text) |
| + .join("") |
| + ); |
| + } catch (e) { |
| + tag(`addressed-classifier: ${e.message}`); |
| + return false; |
| + } |
| } |
| // Build the Claude message list from recent thread history (already includes |
| @@ -221,7 +261,9 @@ export function runBot({ name, session, builtinPersona = "", builtinDocs = "" }) |
| } |
| if (p.name === name) return; // our own message, echoed back |
| if (agentNames.has(p.name)) return; // bots never talk to bots (see loadAgents) |
| - if (!wantsReply(entry.kind, p.text)) return; |
| + const trigger = replyTrigger(entry.kind, p.text); |
| + if (!trigger) return; |
| + if (trigger === "classify" && !(await isAddressed(threadId, entry.key))) return; |
| await reply(threadId, entry); |
| } |
diff --git a/changelog/2026-07-02-bots-reply-when-spoken-to.md b/changelog/2026-07-02-bots-reply-when-spoken-to.md| new file mode 100644 |
| index 0000000..243e09f |
| --- /dev/null |
| +++ b/changelog/2026-07-02-bots-reply-when-spoken-to.md |
| @@ -0,0 +1,4 @@ |
| +Helpers now only join in when you're actually talking to them. @mention one |
| +(like @Claude) for a guaranteed reply; if a message just uses a helper's name |
| +in passing, it first checks whether it's really being spoken to and stays |
| +quiet otherwise. DMs are unchanged — a helper always replies there. |
diff --git a/docs/bots.md b/docs/bots.md| index aa366cd..38729e1 100644 |
| --- a/docs/bots.md |
| +++ b/docs/bots.md |
| @@ -55,7 +55,9 @@ A headless Node sidecar that behaves like a member's browser: |
| - `agent.mjs` — connect `/ws`, unwrap thread keys, on each incoming `msg` |
| decrypt, build context from the thread's recent history, call Claude |
| (`claude-opus-4-8`, adaptive thinking), post an encrypted reply. **DM → replies |
| - to everything; group → only on @mention.** Reconnects with backoff; picks up |
| + to everything; group → @mention always replies, a bare-name mention replies |
| + only if a cheap Haiku yes/no call judges the bot is actually being addressed |
| + (talking *about* a bot doesn't summon it).** Reconnects with backoff; picks up |
| new threads on first unknown `thread_id`. |
| - Server + web: additive `users.is_agent` column, roster `BOT` tag, and a |
| dedicated admin **"Add an agent"** section (distinct from inviting a person) |