amadan
media view: search (incl. on-device photo search), Mac window + zoom, Save (#1480)
* mac: media sheet — link + doc rows highlight under the pointer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * media view: search within a tab — sender, URL/title, filename (web + iOS + Mac) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * ios: photo search — on-device Vision labels + OCR index behind the media search (native-only v1) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * ios: photo index in two passes — labels first, OCR second with its one-off model compile warmed in parallel Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * media view: spinner on progress rows, resizable Mac sheet, Save on every item Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * ios: a photo still decrypting previews its BlurHash ghost, not "Unsupported file format" Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * mac: media view gets its own resizable window, and Save reaches the real ~/Downloads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * mac: pinch to zoom, drag to pan, double-click to toggle in the media viewer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * mac: the feed's photo lightbox zooms too — pinch, pan, double-click Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 * changelog: search inside Media, links and docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GokbEx2gTwsXD8Yvf6jpe7 ---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
19 files changed,
+1186
−131
CLAUDE.md+8 −0changelog/2026-08-29-media-search.md+3 −0docs/superpowers/plans/2026-08-28-media-links-docs-native.md+19 −0ios/LChat/Views/MediaLinksDocsSheet.swift+178 −30ios/LChatCore/Sources/LChatCore/ImageSearchIndex.swift+222 −0ios/LChatCore/Sources/LChatCore/MessageStore.swift+66 −0ios/LChatCore/Sources/LChatCore/ThreadMedia.swift+40 −0ios/LChatCore/Tests/LChatCoreTests/ThreadMediaTests.swift+56 −0ios/LChatMac/LChatMac-Dev.entitlements+5 −0ios/LChatMac/LChatMac.entitlements+9 −0ios/LChatMac/Lightbox.swift+72 −2ios/LChatMac/MediaLinksDocsSheet.swift+262 −78ios/LChatMac/MediaWindows.swift+59 −0ios/LChatMac/NativeChatView.swift+12 −14test/threadmedia.test.mjs+21 −1tools/dev+87 −0web/static/mediaview.js+39 −6web/static/style.css+9 −0web/static/threadmedia.js+19 −0
diff --git a/CLAUDE.md b/CLAUDE.md| index 21243d7..2d94041 100644 |
| --- a/CLAUDE.md |
| +++ b/CLAUDE.md |
| @@ -686,6 +686,14 @@ Simple command-line interface for common dev tasks. Run from anywhere in the rep |
| ./tools/dev build ios # Build native iOS app (xcodebuild) |
| ./tools/dev run mac # Launch the Mac app |
| ./tools/dev run ios # Run iOS app on simulator |
| +./tools/dev install ios [id] # Build + install + launch on a REAL iPhone, over |
| + # the network (no cable): unlocked, trusted, same |
| + # Wi-Fi. Device id defaults to the first paired |
| + # one; LCHAT_IOS_DEVICE overrides. Signing is |
| + # automatic — a new device must be registered |
| + # with the team first. "iOS <v> is not installed" |
| + # = point DEVELOPER_DIR at an Xcode with that |
| + # platform. |
| ./tools/dev alt [slug] # Build + launch a side-by-side dev alt of the Mac app |
| ./tools/dev up [flags] # Start local dev server (persistent config) |
| # + the webhook-ingest sidecar (--no-hooks skips) |
diff --git a/changelog/2026-08-29-media-search.md b/changelog/2026-08-29-media-search.md| new file mode 100644 |
| index 0000000..62c3be5 |
| --- /dev/null |
| +++ b/changelog/2026-08-29-media-search.md |
| @@ -0,0 +1,3 @@ |
| +🤖 "Media, links and docs" now has a search box. Type in it to narrow the open |
| +tab — by who shared something, by a link's address or its page title, or by a |
| +file's name. Clear it with Escape. |
diff --git a/docs/superpowers/plans/2026-08-28-media-links-docs-native.md b/docs/superpowers/plans/2026-08-28-media-links-docs-native.md| index a7e85fe..3f78aeb 100644 |
| --- a/docs/superpowers/plans/2026-08-28-media-links-docs-native.md |
| +++ b/docs/superpowers/plans/2026-08-28-media-links-docs-native.md |
| @@ -82,6 +82,25 @@ tested. |
| - "Show in chat" selects the thread and sets `app.nativeJump` (the search |
| hit's path). |
| +## Search (follow-up, same day) |
| + |
| +- **Search within a tab** — web, iOS, Mac: one pure `filterEntries` / |
| + `ThreadMedia.filtered(query:)` (sender on every tab; URL + already-fetched |
| + unfurl title on Links; filename on Docs), Node- and Swift-tested. |
| +- **Image search — native-only for v1 (Paul, 2026-08-28).** `ImageSearchIndex` |
| + (LChatCore) runs Apple's Vision on device — `VNClassifyImageRequest` scene |
| + labels at Apple's ≥90%-precision idiom, plus `VNRecognizeTextRequest` OCR |
| + (the sleeper hit: screenshots, whiteboards, menus) — over each still once |
| + (videos → poster), persisted per blob in MessageStore's `image_index` |
| + (derived plaintext beside the decrypted payloads; never on a server; |
| + `ver` column so a recipe bump re-indexes). Runs on sheet open, one image |
| + at a time off the main actor, narrated inline with Stop, resumable. Search |
| + then matches what's *in* a photo through the same `filtered(query:)` via |
| + `imageTextFor`. **Why native-only:** the E2EE rule demands on-device |
| + inference; web has no free classifier — parity there means shipping a |
| + model (MobileNet/CLIP via CDN, 5–20 MB, against the no-build-step posture |
| + and the CSP). Revisit when a light model earns its weight. |
| + |
| ## Verification |
| - `swift test` in `ios/LChatCore`: `ThreadMediaTests` (10) green; the one |
diff --git a/ios/LChat/Views/MediaLinksDocsSheet.swift b/ios/LChat/Views/MediaLinksDocsSheet.swift| index 061e110..a230fe5 100644 |
| --- a/ios/LChat/Views/MediaLinksDocsSheet.swift |
| +++ b/ios/LChat/Views/MediaLinksDocsSheet.swift |
| @@ -25,6 +25,14 @@ struct MediaLinksDocsSheet: View { |
| // The sheet's own walker — deliberately not shared with the search UI's |
| // instance-wide runner; the store marks are the shared state. |
| @StateObject private var backfill = HistoryBackfill() |
| + // What's IN each photo (Vision, on device) — search matches "dog" or the |
| + // text on a whiteboard. Indexes the thread's stills on open, resumably. |
| + @StateObject private var imageIndex = ImageSearchIndex() |
| + @State private var query = "" |
| + // Unfurl titles the Links tab has fetched so far, so search matches a |
| + // link by title too (a title only exists once its row was on screen — |
| + // search never triggers fetches of its own). |
| + @State private var linkTitles: [URL: String] = [:] |
| // A tapped tile's collection viewer (Quick Look, staged lazily). |
| @State private var viewer: MediaCollectionItem? |
| struct MediaCollectionItem: Identifiable { |
| @@ -40,15 +48,30 @@ struct MediaLinksDocsSheet: View { |
| let id = UUID() |
| let url: URL |
| } |
| + // A "Save…" from the grid/rows: the staged file behind the system share |
| + // sheet (Save to Photos / Files / AirDrop). |
| + @State private var shareItem: ShareItem? |
| + @State private var shareCleanupDir: URL? |
| + @State private var sharingID: String? |
| + struct ShareItem: Identifiable { |
| + let id = UUID() |
| + let url: URL |
| + } |
| // Muted/blocked senders hide at query time (web hiddenUids) — un-muting |
| // needs no re-extraction. |
| private var hiddenUids: Set<String> { store.mutedPeople.union(store.blockedPeople) } |
| + // Moderation applies always; the query narrows the open tab (and the |
| + // viewer's collection with it, so swipes walk the search results). |
| + private func visible(_ entries: [ThreadMedia.Entry]) -> [ThreadMedia.Entry] { |
| + ThreadMedia.filtered(ThreadMedia.filtered(entries, hiddenUids: hiddenUids), |
| + query: query, titleFor: { linkTitles[$0] }, |
| + imageTextFor: { imageIndex.textFor($0) }) |
| + } |
| + |
| private var counts: (media: Int, links: Int, docs: Int) { |
| - (ThreadMedia.filtered(tabs.media, hiddenUids: hiddenUids).count, |
| - ThreadMedia.filtered(tabs.links, hiddenUids: hiddenUids).count, |
| - ThreadMedia.filtered(tabs.docs, hiddenUids: hiddenUids).count) |
| + (visible(tabs.media).count, visible(tabs.links).count, visible(tabs.docs).count) |
| } |
| var body: some View { |
| @@ -63,24 +86,20 @@ struct MediaLinksDocsSheet: View { |
| .padding(.horizontal, 16) |
| .padding(.vertical, 8) |
| if backfill.running { |
| - // Inline, occupying layout (never a floating toast): the |
| - // walk's narration plus Stop, exactly the web overlay's. |
| - HStack(spacing: 8) { |
| - Text(backfill.progress ?? "Loading earlier messages…") |
| - .font(.footnote) |
| - .foregroundStyle(.secondary) |
| - Spacer(minLength: 0) |
| - Button("Stop") { backfill.cancel() } |
| - .font(.footnote.weight(.semibold)) |
| - } |
| - .padding(.horizontal, 16) |
| - .padding(.bottom, 6) |
| - .accessibilityElement(children: .combine) |
| + progressRow(backfill.progress ?? "Loading earlier messages…") { backfill.cancel() } |
| + } |
| + if imageIndex.running { |
| + progressRow(imageIndex.progress ?? "Indexing photos…") { imageIndex.cancel() } |
| } |
| content |
| } |
| .navigationTitle("Media, links and docs") |
| .navigationBarTitleDisplayMode(.inline) |
| + // Search within the open tab — sender, URL/title, filename |
| + // (ThreadMedia.filtered(query:)), the system bar-drawer field |
| + // the people picker uses. |
| + .searchable(text: $query, placement: .navigationBarDrawer(displayMode: .always), |
| + prompt: "Search") |
| .toolbar { |
| ToolbarItem(placement: .confirmationAction) { |
| Button("Done") { dismiss() } |
| @@ -89,7 +108,10 @@ struct MediaLinksDocsSheet: View { |
| } |
| .task(id: backfill.pageTick) { await reload() } |
| .onAppear { startWalkIfIncomplete() } |
| - .onDisappear { backfill.cancel() } |
| + .onDisappear { |
| + backfill.cancel() |
| + imageIndex.cancel() |
| + } |
| .fullScreenCover(item: $viewer) { item in |
| MediaCollectionPreview(entries: item.entries, index: item.index, |
| load: { await model.blobData($0) }, |
| @@ -100,6 +122,9 @@ struct MediaLinksDocsSheet: View { |
| QuickLookPreview(url: item.url) { docPreview = nil } |
| .ignoresSafeArea() |
| } |
| + .sheet(item: $shareItem, onDismiss: cleanupShare) { item in |
| + ActivitySheet(url: item.url) |
| + } |
| } |
| // "Media · 12" — counts pluralise implicitly (the label is the kind) and |
| @@ -108,18 +133,76 @@ struct MediaLinksDocsSheet: View { |
| n > 0 ? "\(label) · \(n.formatted())" : label |
| } |
| + // Inline, occupying layout (never a floating toast): a walk's narration |
| + // plus its Stop — the history walk and the photo indexer share the shape. |
| + private func progressRow(_ text: String, stop: @escaping () -> Void) -> some View { |
| + HStack(spacing: 8) { |
| + // The spinner is the "still working" signal: a count that sits on |
| + // "0 of 5" while a slow photo decrypts otherwise reads as stuck. |
| + ProgressView() |
| + .controlSize(.mini) |
| + Text(text) |
| + .font(.footnote) |
| + .foregroundStyle(.secondary) |
| + Spacer(minLength: 0) |
| + Button("Stop", action: stop) |
| + .font(.footnote.weight(.semibold)) |
| + } |
| + .padding(.horizontal, 16) |
| + .padding(.bottom, 6) |
| + .accessibilityElement(children: .combine) |
| + } |
| + |
| + // Hand a photo/video/file to the system share sheet — Save to Photos, |
| + // Save to Files, AirDrop. (The tap path's Quick Look carries the same |
| + // button; this is the shortcut from the grid.) Staged like every other |
| + // preview: a temp dir burned when the sheet closes. |
| + private func share(_ att: Attachment) { |
| + guard shareItem == nil else { return } |
| + sharingID = att.id |
| + Task { |
| + defer { sharingID = nil } |
| + guard let data = await model.blobData(att), |
| + let url = stageShareFile( |
| + data, name: AttachmentDownload.safeFilename(name: att.name, mime: att.type)) |
| + else { return } |
| + shareItem = ShareItem(url: url) |
| + } |
| + } |
| + |
| + private func stageShareFile(_ data: Data, name: String) -> URL? { |
| + let dir = FileManager.default.temporaryDirectory |
| + .appendingPathComponent("media-share-\(UUID().uuidString)", isDirectory: true) |
| + do { |
| + try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) |
| + let url = dir.appendingPathComponent(name) |
| + try data.write(to: url, options: .completeFileProtection) |
| + shareCleanupDir = dir |
| + return url |
| + } catch { |
| + return nil |
| + } |
| + } |
| + |
| + private func cleanupShare() { |
| + if let dir = shareCleanupDir { try? FileManager.default.removeItem(at: dir) } |
| + shareCleanupDir = nil |
| + } |
| + |
| @ViewBuilder private var content: some View { |
| - let entries = ThreadMedia.filtered( |
| - tab == "media" ? tabs.media : tab == "links" ? tabs.links : tabs.docs, |
| - hiddenUids: hiddenUids) |
| + let all = tab == "media" ? tabs.media : tab == "links" ? tabs.links : tabs.docs |
| + let entries = visible(all) |
| if entries.isEmpty { |
| Spacer() |
| - if loaded { |
| + if !loaded { |
| + ProgressView() |
| + } else if !ThreadMedia.filtered(all, hiddenUids: hiddenUids).isEmpty { |
| + Text("Nothing matches.") |
| + .foregroundStyle(.secondary) |
| + } else { |
| Text(tab == "media" ? "No photos or videos shared yet." |
| : tab == "links" ? "No links shared yet." : "No files shared yet.") |
| .foregroundStyle(.secondary) |
| - } else { |
| - ProgressView() |
| } |
| Spacer() |
| } else { |
| @@ -157,6 +240,9 @@ struct MediaLinksDocsSheet: View { |
| .accessibilityLabel("\(att.isVideo ? "Video" : "Photo") from \(e.name.isEmpty ? "someone" : e.name), \(relTime(e.ts / 1000))") |
| .accessibilityAddTraits(.isButton) |
| .contextMenu { |
| + Button { share(att) } label: { |
| + Label("Save…", systemImage: "square.and.arrow.down") |
| + } |
| Button { onJump(e.sid) } label: { |
| Label("Show in chat", systemImage: "arrow.uturn.left") |
| } |
| @@ -177,8 +263,15 @@ struct MediaLinksDocsSheet: View { |
| private func linkRow(_ e: ThreadMedia.Entry) -> some View { |
| LinkUnfurlRow(e: e, unfurl: { await model.unfurl($0) }, |
| - peek: { model.cachedUnfurl($0) }) |
| + peek: { model.cachedUnfurl($0) }, |
| + onTitle: { url, title in linkTitles[url] = title }) |
| .contextMenu { |
| + if let url = e.url { |
| + Button { UIPasteboard.general.string = url.absoluteString } label: { |
| + Label("Copy Link", systemImage: "doc.on.doc") |
| + } |
| + ShareLink(item: url) { Label("Share…", systemImage: "square.and.arrow.up") } |
| + } |
| Button { onJump(e.sid) } label: { |
| Label("Show in chat", systemImage: "arrow.uturn.left") |
| } |
| @@ -200,7 +293,9 @@ struct MediaLinksDocsSheet: View { |
| .lineLimit(1) |
| } |
| Spacer(minLength: 0) |
| - if docBusy == e.att?.id { ProgressView().controlSize(.small) } |
| + if docBusy == e.att?.id || sharingID == e.att?.id { |
| + ProgressView().controlSize(.small) |
| + } |
| } |
| .padding(.horizontal, 16) |
| .padding(.vertical, 6) |
| @@ -208,6 +303,11 @@ struct MediaLinksDocsSheet: View { |
| } |
| .buttonStyle(.plain) |
| .contextMenu { |
| + if let att = e.att { |
| + Button { share(att) } label: { |
| + Label("Save…", systemImage: "square.and.arrow.down") |
| + } |
| + } |
| Button { onJump(e.sid) } label: { |
| Label("Show in chat", systemImage: "arrow.uturn.left") |
| } |
| @@ -271,6 +371,9 @@ struct MediaLinksDocsSheet: View { |
| oldestSID = first.id // page() returns ascending, so first = oldest |
| } |
| loaded = true |
| + // Every photo the thread now holds gets looked at once (on device). |
| + imageIndex.ensure(instance: instance.id, entries: tabs.media, |
| + load: { await model.blobData($0) }) |
| } |
| // Auto-complete on open (spec § Completeness): quiet when the walk has |
| @@ -286,6 +389,19 @@ struct MediaLinksDocsSheet: View { |
| } |
| } |
| +// 🤖 The system share sheet over a staged file — Save to Photos, Save to |
| +// Files, AirDrop. SwiftUI's ShareLink needs its URL up front, and ours only |
| +// exists after an async decrypt, so the item drives a sheet instead. |
| +private struct ActivitySheet: UIViewControllerRepresentable { |
| + let url: URL |
| + |
| + func makeUIViewController(context: Context) -> UIActivityViewController { |
| + UIActivityViewController(activityItems: [url], applicationActivities: nil) |
| + } |
| + |
| + func updateUIViewController(_ controller: UIActivityViewController, context: Context) {} |
| +} |
| + |
| // One grid tile: BlurHash ghost immediately (the payload carries it), the |
| // decrypted thumbnail when loaded — MessageBubble.AttachmentImage's recipe at |
| // grid size. LazyVGrid only materialises visible tiles, so decryption stays |
| @@ -338,6 +454,7 @@ private struct LinkUnfurlRow: View { |
| let e: ThreadMedia.Entry |
| let unfurl: (URL) async -> LinkPreview? |
| let peek: (URL) -> LinkPreview?? |
| + let onTitle: (URL, String) -> Void // feeds the sheet's search-by-title memo |
| @Environment(\.openURL) private var openURL |
| @State private var title: String? |
| @@ -366,8 +483,8 @@ private struct LinkUnfurlRow: View { |
| .buttonStyle(.plain) |
| .task(id: e.id) { |
| guard let url = e.url else { return } |
| - if case .some(let hit) = peek(url) { title = hit?.title; return } |
| - title = await unfurl(url)?.title |
| + if case .some(let hit) = peek(url) { title = hit?.title } else { title = await unfurl(url)?.title } |
| + if let title, !title.isEmpty { onTitle(url, title) } |
| } |
| } |
| } |
| @@ -409,6 +526,7 @@ private struct MediaCollectionPreview: UIViewControllerRepresentable { |
| private let onClose: () -> Void |
| private let dir: URL |
| private var staged: [Int: URL] = [:] |
| + private var placeholders: [Int: URL] = [:] // BlurHash ghosts, shown while decrypting |
| private var staging: Set<Int> = [] |
| private var indexWatch: NSKeyValueObservation? |
| private weak var controller: QLPreviewController? |
| @@ -482,9 +600,39 @@ private struct MediaCollectionPreview: UIViewControllerRepresentable { |
| func previewController(_ controller: QLPreviewController, |
| previewItemAt index: Int) -> QLPreviewItem { |
| ensure(index) |
| - // A nil URL renders Quick Look's own loading placeholder until |
| - // refreshCurrentPreviewItem swaps the staged file in. |
| - return LazyItem(url: staged[index]) |
| + // Quick Look has no loading state: handed a nil URL it flashes |
| + // "Unsupported file format". So an item that hasn't finished |
| + // decrypting previews its BlurHash ghost — the same blur the grid |
| + // tile shows — and refreshCurrentPreviewItem swaps in the real |
| + // file the moment the bytes land. |
| + return LazyItem(url: staged[index] ?? placeholder(index)) |
| + } |
| + |
| + // The ghost file for an item, written once: its BlurHash at a size |
| + // Quick Look can scale up, or a plain dark tile when the payload |
| + // carries no hash. Same temp dir, so it burns with everything else. |
| + private func placeholder(_ i: Int) -> URL? { |
| + if let hit = placeholders[i] { return hit } |
| + guard entries.indices.contains(i) else { return nil } |
| + let att = entries[i].stillAttachment ?? entries[i].att |
| + let url = dir.appendingPathComponent("\(i)-loading.jpg") |
| + // Match the photo's aspect so the blur doesn't jump on swap. |
| + let w = 48 |
| + let h = max(1, Int((Double(att?.h ?? 3) / Double(max(att?.w ?? 4, 1)) * Double(w)).rounded())) |
| + var image: UIImage? |
| + if let bh = att?.bh, let cg = Blurhash.decode(bh, width: w, height: h) { |
| + image = UIImage(cgImage: cg) |
| + } else { |
| + let renderer = UIGraphicsImageRenderer(size: CGSize(width: w, height: h)) |
| + image = renderer.image { ctx in |
| + UIColor.secondarySystemBackground.setFill() |
| + ctx.fill(CGRect(x: 0, y: 0, width: w, height: h)) |
| + } |
| + } |
| + guard let data = image?.jpegData(compressionQuality: 0.8), |
| + (try? data.write(to: url, options: .completeFileProtection)) != nil else { return nil } |
| + placeholders[i] = url |
| + return url |
| } |
| @objc func done() { onClose() } |
diff --git a/ios/LChatCore/Sources/LChatCore/ImageSearchIndex.swift b/ios/LChatCore/Sources/LChatCore/ImageSearchIndex.swift| new file mode 100644 |
| index 0000000..e9299b5 |
| --- /dev/null |
| +++ b/ios/LChatCore/Sources/LChatCore/ImageSearchIndex.swift |
| @@ -0,0 +1,222 @@ |
| +import Foundation |
| +import Vision |
| + |
| +// 🤖 Photos-style search over a chat's shared pictures — native-first, and |
| +// deliberately so: Apple's Vision framework runs ON DEVICE, so the decrypted |
| +// bytes never leave the machine that already holds them, and the E2EE rule |
| +// holds exactly as it does for the feed rendering the same photos. (Web has |
| +// no free on-device classifier — a model download is the price of parity |
| +// there; scoped "native-only for v1", docs/superpowers/plans/ |
| +// 2026-08-28-media-links-docs-native.md § Image search.) |
| +// |
| +// Two passes over each still (videos → poster), persisted per blob in |
| +// MessageStore's image_index (derived plaintext beside the decrypted payloads |
| +// it came from — never on a server), each row stamped with how far it got: |
| +// |
| +// pass 1, ver 1 — scene/object classification (Vision's ~1,300 labels: |
| +// "dog", "beach", "receipt"). ~0.1s a photo, so "dog" works within |
| +// seconds of opening. |
| +// pass 2, ver 2 — accurate text recognition, the sleeper hit in a chat |
| +// (screenshots, whiteboards, menus, boarding passes). ~0.2s a photo once |
| +// warm — but Vision COMPILES the text model on its first use per machine |
| +// (measured: 58s on an M-series Mac, 0.2s ever after), so the pass runs |
| +// second and the compile is warmed in parallel with pass 1, never in |
| +// front of the counter. (Seen as "Indexing photos… 0 of 251" for a |
| +// minute, 2026-08-29, before this split.) |
| +// |
| +// Proportional to the thread's photos, narrated inline with Stop, resumable |
| +// (every row lands as it's done), one image at a time off the main actor. |
| +@MainActor |
| +public final class ImageSearchIndex: ObservableObject { |
| + /// A row at this version is complete; bump when the recipe changes so |
| + /// older rows re-index on next open. |
| + public static let version = 2 |
| + static let labelsVersion = 1 |
| + |
| + @Published public private(set) var running = false |
| + @Published public private(set) var progress: String? |
| + /// Bumps as each photo's row lands — search UIs re-filter on it. |
| + @Published public private(set) var tick = 0 |
| + |
| + private var instance: String? |
| + private var rows: [String: MessageStore.ImageIndexRow] = [:] // blob id → what's in it |
| + private var searchText: [String: String] = [:] // blob id → lowercased haystack |
| + private var labelQueue: [Attachment] = [] |
| + private var textQueue: [Attachment] = [] |
| + private var queued: Set<String> = [] |
| + private var loaded = false |
| + private var task: Task<Void, Never>? |
| + private static var ocrWarmed = false |
| + |
| + public init() {} |
| + |
| + /// The indexed haystack for a still (labels + recognised text so far, |
| + /// lowercased), nil until that photo has been looked at. |
| + public func textFor(_ blobID: String) -> String? { searchText[blobID] } |
| + |
| + /// Everything known about a still, for UI that wants to show labels. |
| + public func row(_ blobID: String) -> MessageStore.ImageIndexRow? { rows[blobID] } |
| + |
| + /// Queue every media entry's still that isn't fully indexed yet and start |
| + /// (or keep) the worker draining. Re-entrant: each backfill page calls |
| + /// this with the grown list and only the new stills enqueue. |
| + public func ensure(instance inst: String, entries: [ThreadMedia.Entry], |
| + load: @escaping (Attachment) async -> Data?) { |
| + if instance != inst { |
| + instance = inst |
| + rows = [:] |
| + searchText = [:] |
| + labelQueue = [] |
| + textQueue = [] |
| + queued = [] |
| + loaded = false |
| + } |
| + let stills = entries.compactMap { $0.stillAttachment } |
| + Task { |
| + // One read of the instance's persisted index on first use; the |
| + // in-memory map is the working copy from then on. |
| + if !loaded { |
| + let known = await MessageStore.shared.imageIndex(instance: inst, minVersion: Self.labelsVersion) |
| + guard instance == inst else { return } |
| + for (blob, row) in known { adopt(blob, row) } |
| + loaded = true |
| + } |
| + var fresh = 0 |
| + for still in stills where !queued.contains(still.id) { |
| + let ver = rows[still.id]?.ver ?? 0 |
| + if ver >= Self.version { continue } |
| + queued.insert(still.id) |
| + if ver >= Self.labelsVersion { textQueue.append(still) } else { labelQueue.append(still) } |
| + fresh += 1 |
| + } |
| + guard fresh > 0 else { return } |
| + startIfIdle(load: load) |
| + } |
| + } |
| + |
| + public func cancel() { |
| + task?.cancel() |
| + } |
| + |
| + private func adopt(_ blob: String, _ row: MessageStore.ImageIndexRow) { |
| + rows[blob] = row |
| + searchText[blob] = Self.haystack(labels: row.labels, text: row.text) |
| + } |
| + |
| + private func startIfIdle(load: @escaping (Attachment) async -> Data?) { |
| + guard task == nil, let inst = instance else { return } |
| + running = true |
| + progress = Self.progressLabel(reading: labelQueue.isEmpty, done: 0, |
| + total: labelQueue.isEmpty ? textQueue.count : labelQueue.count) |
| + // Warm the text model's one-off compile while pass 1 runs, so pass 2 |
| + // starts at full speed instead of parking on its first photo. |
| + if !Self.ocrWarmed { |
| + Self.ocrWarmed = true |
| + Task.detached(priority: .utility) { Self.warmOCR() } |
| + } |
| + task = Task { [weak self] in |
| + await self?.drain(instance: inst, load: load) |
| + await MainActor.run { |
| + guard let self else { return } |
| + self.running = false |
| + self.progress = nil |
| + self.task = nil |
| + // A cancelled pass leaves its unvisited stills queued; the |
| + // next ensure() picks them straight back up. |
| + } |
| + } |
| + } |
| + |
| + private func drain(instance inst: String, load: @escaping (Attachment) async -> Data?) async { |
| + // Pass 1 for everything waiting, then pass 2 — a still that arrives |
| + // mid-pass-2 (a backfill page) gets its labels first, then its text. |
| + while !Task.isCancelled, !labelQueue.isEmpty || !textQueue.isEmpty { |
| + let reading = labelQueue.isEmpty |
| + let total = reading ? textQueue.count : labelQueue.count |
| + var done = 0 |
| + while !Task.isCancelled, let still = reading ? textQueue.first : labelQueue.first { |
| + if reading { textQueue.removeFirst() } else { labelQueue.removeFirst() } |
| + progress = Self.progressLabel(reading: reading, done: done, total: total) |
| + if let data = await load(still) { |
| + // Vision is CPU/ANE-bound and synchronous — never on the |
| + // main actor. One image at a time keeps memory flat. |
| + let prior = rows[still.id] |
| + let row = await Task.detached(priority: .utility) { |
| + reading ? Self.readText(data, labels: prior?.labels ?? []) : Self.classify(data) |
| + }.value |
| + if Task.isCancelled { break } |
| + await MessageStore.shared.setImageIndex(instance: inst, blob: still.id, row: row) |
| + guard instance == inst else { return } |
| + adopt(still.id, row) |
| + if !reading { textQueue.append(still) } else { queued.remove(still.id) } |
| + tick += 1 |
| + } else { |
| + queued.remove(still.id) |
| + } |
| + done += 1 |
| + progress = Self.progressLabel(reading: reading, done: done, total: total) |
| + if !reading, labelQueue.isEmpty { break } // pass 1 drained: on to the text |
| + } |
| + } |
| + } |
| + |
| + // ---- the recipe (pure, off-actor) ------------------------------------ |
| + |
| + /// Pass 1: what's in the picture. A failure (undecodable image, an |
| + /// unsupported format) indexes as empty rather than retrying forever. |
| + nonisolated static func classify(_ data: Data) -> MessageStore.ImageIndexRow { |
| + let req = VNClassifyImageRequest() |
| + try? VNImageRequestHandler(data: data, options: [:]).perform([req]) |
| + // Apple's own idiom for "labels a person would agree with": keep an |
| + // observation only where the model is ≥90% precise at ≥1% recall. |
| + let labels = (req.results ?? []) |
| + .filter { $0.hasMinimumRecall(0.01, forPrecision: 0.9) } |
| + .map { normaliseLabel($0.identifier) } |
| + return MessageStore.ImageIndexRow(labels: labels, text: "", ver: labelsVersion) |
| + } |
| + |
| + /// Pass 2: the words in the picture, merged onto pass 1's labels. |
| + nonisolated static func readText(_ data: Data, labels: [String]) -> MessageStore.ImageIndexRow { |
| + let req = VNRecognizeTextRequest() |
| + req.recognitionLevel = .accurate |
| + req.usesLanguageCorrection = true |
| + try? VNImageRequestHandler(data: data, options: [:]).perform([req]) |
| + let text = (req.results ?? []) |
| + .compactMap { $0.topCandidates(1).first?.string } |
| + .joined(separator: "\n") |
| + return MessageStore.ImageIndexRow(labels: labels, text: text, ver: version) |
| + } |
| + |
| + /// Trip the text model's compile on a 1×1 image so the real pass never |
| + /// waits for it. Vision caches the compiled model on disk, so this is |
| + /// cheap on every launch but the first on a machine. |
| + nonisolated static func warmOCR() { |
| + var px: [UInt8] = [255, 255, 255, 255] |
| + let ctx = px.withUnsafeMutableBytes { buf in |
| + CGContext(data: buf.baseAddress, width: 1, height: 1, bitsPerComponent: 8, bytesPerRow: 4, |
| + space: CGColorSpaceCreateDeviceRGB(), |
| + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue) |
| + } |
| + guard let cg = ctx?.makeImage() else { return } |
| + let req = VNRecognizeTextRequest() |
| + req.recognitionLevel = .accurate |
| + try? VNImageRequestHandler(cgImage: cg, options: [:]).perform([req]) |
| + } |
| + |
| + /// Vision identifiers read "birthday_cake"; people type "birthday cake". |
| + nonisolated static func normaliseLabel(_ id: String) -> String { |
| + id.replacingOccurrences(of: "_", with: " ").lowercased() |
| + } |
| + |
| + /// The lowercased haystack the media search substring-matches. |
| + nonisolated static func haystack(labels: [String], text: String) -> String { |
| + (labels.joined(separator: " ") + "\n" + text).lowercased() |
| + } |
| + |
| + /// "Indexing photos… 12 of 340" / "Reading text in photos… 12 of 340" — |
| + /// counts localised. |
| + nonisolated static func progressLabel(reading: Bool, done: Int, total: Int) -> String { |
| + let verb = reading ? "Reading text in photos…" : "Indexing photos…" |
| + return total > 0 ? "\(verb) \(done.formatted()) of \(total.formatted())" : verb |
| + } |
| +} |
diff --git a/ios/LChatCore/Sources/LChatCore/MessageStore.swift b/ios/LChatCore/Sources/LChatCore/MessageStore.swift| index 0027a66..218583c 100644 |
| --- a/ios/LChatCore/Sources/LChatCore/MessageStore.swift |
| +++ b/ios/LChatCore/Sources/LChatCore/MessageStore.swift |
| @@ -139,6 +139,22 @@ public actor MessageStore { |
| PRIMARY KEY(instance, thread) |
| ) |
| """) |
| + // What's IN each shared photo (ImageSearchIndex): Vision's scene labels |
| + // + recognised text, per blob, so the media view can search photos by |
| + // content. Derived plaintext — it lives here, in the protected store |
| + // beside the decrypted payloads it came from, and never on a server. |
| + // `ver` is the indexer's recipe version, so a smarter pass can re-do |
| + // only the rows an older one produced. |
| + exec(handle, """ |
| + CREATE TABLE IF NOT EXISTS image_index( |
| + instance TEXT NOT NULL, |
| + blob TEXT NOT NULL, |
| + labels TEXT NOT NULL DEFAULT '', |
| + text TEXT NOT NULL DEFAULT '', |
| + ver INTEGER NOT NULL DEFAULT 1, |
| + PRIMARY KEY(instance, blob) |
| + ) |
| + """) |
| // Never in a backup; protected at rest like the keys. |
| try? FileManager.default.setAttributes( |
| [.protectionKey: FileProtectionType.completeUntilFirstUserAuthentication], |
| @@ -417,6 +433,56 @@ public actor MessageStore { |
| (SELECT rowid FROM messages WHERE instance=?) |
| """) { bind($0, 1, instance) } |
| run(handle, "DELETE FROM messages WHERE instance=?") { bind($0, 1, instance) } |
| + run(handle, "DELETE FROM image_index WHERE instance=?") { bind($0, 1, instance) } |
| + } |
| + |
| + // ---- image index (ImageSearchIndex) -------------------------------------- |
| + |
| + public struct ImageIndexRow: Sendable { |
| + public let labels: [String] |
| + public let text: String |
| + /// How far the indexer got with this photo (ImageSearchIndex's pass |
| + /// versions): labels-only rows are usable and get their text later. |
| + public let ver: Int |
| + public init(labels: [String], text: String, ver: Int = 1) { |
| + self.labels = labels |
| + self.text = text |
| + self.ver = ver |
| + } |
| + } |
| + |
| + /// Every indexed photo for an instance, by blob id — one read at open, |
| + /// held in memory by the index (a thread's photos are a subset). |
| + public func imageIndex(instance: String, minVersion: Int = 1) -> [String: ImageIndexRow] { |
| + guard let handle = open() else { return [:] } |
| + var out: [String: ImageIndexRow] = [:] |
| + var stmt: OpaquePointer? |
| + guard sqlite3_prepare_v2(handle, |
| + "SELECT blob, labels, text, ver FROM image_index WHERE instance=? AND ver>=?", |
| + -1, &stmt, nil) == SQLITE_OK else { return [:] } |
| + defer { sqlite3_finalize(stmt) } |
| + bind(stmt, 1, instance) |
| + sqlite3_bind_int64(stmt, 2, Int64(minVersion)) |
| + while sqlite3_step(stmt) == SQLITE_ROW { |
| + guard let blob = sqlite3_column_text(stmt, 0).map({ String(cString: $0) }) else { continue } |
| + let labels = sqlite3_column_text(stmt, 1).map { String(cString: $0) } ?? "" |
| + let text = sqlite3_column_text(stmt, 2).map { String(cString: $0) } ?? "" |
| + out[blob] = ImageIndexRow(labels: labels.split(separator: "\n").map(String.init), text: text, |
| + ver: Int(sqlite3_column_int64(stmt, 3))) |
| + } |
| + return out |
| + } |
| + |
| + public func setImageIndex(instance: String, blob: String, row: ImageIndexRow) { |
| + guard let handle = open() else { return } |
| + run(handle, """ |
| + INSERT INTO image_index(instance, blob, labels, text, ver) VALUES(?,?,?,?,?) |
| + ON CONFLICT(instance, blob) DO UPDATE SET labels=excluded.labels, text=excluded.text, ver=excluded.ver |
| + """) { |
| + bind($0, 1, instance); bind($0, 2, blob) |
| + bind($0, 3, row.labels.joined(separator: "\n")); bind($0, 4, row.text) |
| + sqlite3_bind_int64($0, 5, Int64(row.ver)) |
| + } |
| } |
| // ---- plumbing ----------------------------------------------------------- |
diff --git a/ios/LChatCore/Sources/LChatCore/ThreadMedia.swift b/ios/LChatCore/Sources/LChatCore/ThreadMedia.swift| index 1be5e08..8a92d33 100644 |
| --- a/ios/LChatCore/Sources/LChatCore/ThreadMedia.swift |
| +++ b/ios/LChatCore/Sources/LChatCore/ThreadMedia.swift |
| @@ -29,6 +29,22 @@ public enum ThreadMedia { |
| // An attachment id is unique per upload; a repeated URL in one message |
| // is two rows (the web extractor's rule), so links key on position. |
| public var id: String { att.map { "a\(sid):\($0.id)" } ?? "l\(sid):\(url?.absoluteString ?? "")" } |
| + |
| + /// The still to show/index for a media entry: a photo is itself, a |
| + /// video is its poster frame (whole videos never decrypt for a |
| + /// thumbnail or a label — the app's standing rule). |
| + public var stillAttachment: Attachment? { |
| + guard let att, att.isImage || att.isVideo else { return nil } |
| + if att.isVideo { |
| + guard let poster = att.poster, !poster.isEmpty else { return nil } |
| + var still = att |
| + still.id = poster |
| + still.type = "image/jpeg" |
| + still.poster = nil |
| + return still |
| + } |
| + return att |
| + } |
| } |
| public struct Tabs: Sendable { |
| @@ -82,6 +98,30 @@ public enum ThreadMedia { |
| hiddenUids.isEmpty ? entries : entries.filter { !hiddenUids.contains($0.uid) } |
| } |
| + /// Search within a tab (web threadmedia filterEntries): a case-insensitive |
| + /// substring over what each entry actually shows — the sender's name |
| + /// (every tab), the URL and its unfurl title (links; `titleFor` returns |
| + /// the title if it's been fetched, else nil), the filename (docs, and |
| + /// media that carry one), and — native only — what's IN a photo: |
| + /// `imageTextFor(stillBlobID)` returns the indexed labels + recognised |
| + /// text (ImageSearchIndex), already lowercased. Blank query = everything. |
| + public static func filtered(_ entries: [Entry], query: String, |
| + titleFor: (URL) -> String? = { _ in nil }, |
| + imageTextFor: (String) -> String? = { _ in nil }) -> [Entry] { |
| + let q = query.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() |
| + guard !q.isEmpty else { return entries } |
| + return entries.filter { e in |
| + if e.name.lowercased().contains(q) { return true } |
| + if let url = e.url { |
| + if url.absoluteString.lowercased().contains(q) { return true } |
| + if let t = titleFor(url), t.lowercased().contains(q) { return true } |
| + } |
| + if let n = e.att?.name, n.lowercased().contains(q) { return true } |
| + if let still = e.stillAttachment, let t = imageTextFor(still.id), t.contains(q) { return true } |
| + return false |
| + } |
| + } |
| + |
| /// "August 2026" — the month section header (web threadmedia monthLabel). |
| public static func monthLabel(_ tsMs: Int64) -> String { |
| Date(timeIntervalSince1970: TimeInterval(tsMs) / 1000) |
diff --git a/ios/LChatCore/Tests/LChatCoreTests/ThreadMediaTests.swift b/ios/LChatCore/Tests/LChatCoreTests/ThreadMediaTests.swift| index e40688d..ccc3944 100644 |
| --- a/ios/LChatCore/Tests/LChatCoreTests/ThreadMediaTests.swift |
| +++ b/ios/LChatCore/Tests/LChatCoreTests/ThreadMediaTests.swift |
| @@ -107,6 +107,62 @@ final class ThreadMediaTests: XCTestCase { |
| XCTAssertEqual(buckets[1].items.first?.sid, 1) |
| } |
| + func testQueryFilterMatchesSenderURLTitleAndFilename() { |
| + // Mirrors test/threadmedia.test.mjs "filterEntries matches sender, |
| + // URL, fetched unfurl title, and filename". |
| + let mk = { (sid: Int64, name: String, url: String?, file: String?) -> ThreadMedia.Entry in |
| + ThreadMedia.Entry(sid: sid, ts: 1, uid: "u", name: name, |
| + att: file.map { Attachment(id: "b\(sid)", name: $0, type: nil, w: nil, h: nil, |
| + size: nil, poster: nil, dur: nil, bh: nil) }, |
| + url: url.flatMap { URL(string: $0) }) |
| + } |
| + let entries = [ |
| + mk(1, "Ada", "https://example.com/Report", nil), |
| + mk(2, "Bob", "https://news.example/x", nil), |
| + mk(3, "Cy", nil, "Budget-2026.xlsx"), |
| + mk(4, "Ada", nil, nil), |
| + ] |
| + let titles = ["https://news.example/x": "Rain expected Friday"] |
| + let titleFor = { (u: URL) in titles[u.absoluteString] } |
| + let ids = { (q: String) in ThreadMedia.filtered(entries, query: q, titleFor: titleFor).map { $0.sid } } |
| + XCTAssertEqual(ids(""), [1, 2, 3, 4], "blank query = everything") |
| + XCTAssertEqual(ids(" "), [1, 2, 3, 4]) |
| + XCTAssertEqual(ids("ada"), [1, 4], "sender name, any tab") |
| + XCTAssertEqual(ids("REPORT"), [1], "URL substring, case-insensitive") |
| + XCTAssertEqual(ids("friday"), [2], "an already-fetched unfurl title") |
| + XCTAssertEqual(ids("budget"), [3], "doc filename") |
| + XCTAssertEqual(ids("zzz"), []) |
| + XCTAssertEqual(ThreadMedia.filtered(entries, query: "friday").map { $0.sid }, [], |
| + "no titleFor → titles don't match") |
| + } |
| + |
| + func testQueryFilterMatchesIndexedPhotoContent() { |
| + // A photo matches by what's IN it (ImageSearchIndex haystack) — and a |
| + // video by its poster still's index, never its own blob id. |
| + let photo = ThreadMedia.Entry(sid: 1, ts: 1, uid: "u", name: "Ada", |
| + att: Attachment(id: "p1", name: nil, type: "image/jpeg", w: nil, h: nil, |
| + size: nil, poster: nil, dur: nil, bh: nil), url: nil) |
| + let video = ThreadMedia.Entry(sid: 2, ts: 1, uid: "u", name: "Bob", |
| + att: Attachment(id: "v1", name: nil, type: "video/mp4", w: nil, h: nil, |
| + size: nil, poster: "v1-poster", dur: 3, bh: nil), url: nil) |
| + let index = ["p1": ImageSearchIndex.haystack(labels: ["dog", "beach"], text: "Wi-Fi: seaside2026"), |
| + "v1-poster": ImageSearchIndex.haystack(labels: ["birthday cake"], text: "")] |
| + let ids = { (q: String) in |
| + ThreadMedia.filtered([photo, video], query: q, imageTextFor: { index[$0] }).map { $0.sid } |
| + } |
| + XCTAssertEqual(ids("dog"), [1], "a scene label") |
| + XCTAssertEqual(ids("SEASIDE"), [1], "recognised text, case-insensitive") |
| + XCTAssertEqual(ids("birthday cake"), [2], "the video's poster index, keyed by the poster blob") |
| + XCTAssertEqual(ids("v1"), [], "blob ids are never searchable text") |
| + XCTAssertEqual(video.stillAttachment?.id, "v1-poster") |
| + XCTAssertEqual(video.stillAttachment?.type, "image/jpeg") |
| + XCTAssertEqual(ImageSearchIndex.normaliseLabel("Birthday_Cake"), "birthday cake") |
| + XCTAssertEqual(ImageSearchIndex.progressLabel(reading: false, done: 12, total: 1340), |
| + "Indexing photos… 12 of 1,340") |
| + XCTAssertEqual(ImageSearchIndex.progressLabel(reading: true, done: 0, total: 3), |
| + "Reading text in photos… 0 of 3") |
| + } |
| + |
| func testAllLinksMatchesFirstLinkSemantics() { |
| // allLinks (RichText) backs the extractor: every href, in order, |
| // fenced code skipped, quote blocks included — firstLink's rules. |
diff --git a/ios/LChatMac/LChatMac-Dev.entitlements b/ios/LChatMac/LChatMac-Dev.entitlements| index f755d9e..182b4ba 100644 |
| --- a/ios/LChatMac/LChatMac-Dev.entitlements |
| +++ b/ios/LChatMac/LChatMac-Dev.entitlements |
| @@ -30,6 +30,11 @@ |
| <true/> |
| <key>com.apple.security.files.user-selected.read-write</key> |
| <true/> |
| + <!-- Saving an attachment writes to the REAL ~/Downloads; without it the |
| + sandbox silently redirects to the container's own Downloads (see |
| + LChatMac.entitlements for the full note). --> |
| + <key>com.apple.security.files.downloads.read-write</key> |
| + <true/> |
| <key>keychain-access-groups</key> |
| <array> |
| <string>$(AppIdentifierPrefix)com.elevenmessenger.messenger.shared</string> |
diff --git a/ios/LChatMac/LChatMac.entitlements b/ios/LChatMac/LChatMac.entitlements| index 1a4aee7..747e207 100644 |
| --- a/ios/LChatMac/LChatMac.entitlements |
| +++ b/ios/LChatMac/LChatMac.entitlements |
| @@ -17,6 +17,15 @@ |
| downloads — both user-initiated through the powerbox. --> |
| <key>com.apple.security.files.user-selected.read-write</key> |
| <true/> |
| + <!-- Saving an attachment writes to the REAL ~/Downloads. Without this, |
| + FileManager's .downloadsDirectory silently resolves to the sandbox |
| + container's own Downloads (~/Library/Containers/…/Data/Downloads), so |
| + every "save" lands somewhere the member never looks and the Finder |
| + reveal opens a folder that isn't theirs — the feed's file download |
| + (NativeFeedView.save) and the media view's Save both did exactly that |
| + until this line existed. --> |
| + <key>com.apple.security.files.downloads.read-write</key> |
| + <true/> |
| <!-- Same app-family group as iOS: the saved-instances store |
| (LChatCore/Keychain.swift) writes here once the entitlement exists. --> |
| <key>keychain-access-groups</key> |
diff --git a/ios/LChatMac/Lightbox.swift b/ios/LChatMac/Lightbox.swift| index 908b055..df61d73 100644 |
| --- a/ios/LChatMac/Lightbox.swift |
| +++ b/ios/LChatMac/Lightbox.swift |
| @@ -7,6 +7,11 @@ import AVKit |
| // actual size and only scaled down when it would exceed the viewport (with a |
| // margin). Held in memory — deliberately NOT QuickLook, which would silently |
| // write the decrypted bytes to disk. |
| +// |
| +// Pinch to zoom, drag to pan, double-click to toggle — hand-rolled because |
| +// not-QuickLook means not inheriting them (iOS's feed taps DO go through |
| +// Quick Look, so it has all three already; this keeps the two in step). The |
| +// media view's viewer carries the same gestures. |
| struct LightboxItem: Identifiable { |
| let id: String // blob id |
| let name: String? |
| @@ -21,8 +26,15 @@ struct LightboxOverlay: View { |
| // Drives both the scrim fade and the zoom; flipped on with a spring in |
| // onAppear, off + delayed teardown on close (so the zoom-back is visible). |
| @State private var shown = false |
| + // Live gesture values plus what they settle to, so a second pinch picks |
| + // up where the last left off. |
| + @State private var scale: CGFloat = 1 |
| + @State private var steadyScale: CGFloat = 1 |
| + @State private var offset: CGSize = .zero |
| + @State private var steadyOffset: CGSize = .zero |
| private let margin: CGFloat = 40 |
| + private let maxScale: CGFloat = 8 |
| var body: some View { |
| GeometryReader { geo in |
| @@ -39,9 +51,16 @@ struct LightboxOverlay: View { |
| height: shown ? target.height : src.height) |
| .clipShape(RoundedRectangle(cornerRadius: shown ? 6 : 18, style: .continuous)) |
| .shadow(color: .black.opacity(shown ? 0.5 : 0), radius: 30, y: 8) |
| + .scaleEffect(scale) |
| .position(x: shown ? target.midX : src.midX, |
| y: shown ? target.midY : src.midY) |
| - .onTapGesture { close() } |
| + .offset(offset) |
| + .gesture(magnify.simultaneously(with: pan)) |
| + .onTapGesture(count: 2) { toggleZoom() } |
| + // A single click still dismisses — but only at 1×, or |
| + // every click while reading a zoomed screenshot would |
| + // close the photo out from under you. |
| + .onTapGesture { if scale <= 1 { close() } } |
| if shown { |
| VStack { |
| @@ -82,8 +101,59 @@ struct LightboxOverlay: View { |
| return CGRect(x: (viewport.width - w) / 2, y: (viewport.height - h) / 2, width: w, height: h) |
| } |
| + // Trackpad pinch. Clamped at 1 (fitted) so a photo can't shrink away. |
| + private var magnify: some Gesture { |
| + MagnifyGesture() |
| + .onChanged { value in |
| + scale = min(max(steadyScale * value.magnification, 1), maxScale) |
| + } |
| + .onEnded { _ in |
| + steadyScale = scale |
| + if scale <= 1 { resetPan() } |
| + } |
| + } |
| + |
| + // Drag pans, but only while zoomed — at 1× a drag would slide the fitted |
| + // photo off its own scrim. |
| + private var pan: some Gesture { |
| + DragGesture() |
| + .onChanged { value in |
| + guard scale > 1 else { return } |
| + offset = CGSize(width: steadyOffset.width + value.translation.width, |
| + height: steadyOffset.height + value.translation.height) |
| + } |
| + .onEnded { _ in steadyOffset = offset } |
| + } |
| + |
| + private func toggleZoom() { |
| + withAnimation(.easeOut(duration: 0.18)) { |
| + if scale > 1 { |
| + resetZoom() |
| + } else { |
| + scale = 2.5 |
| + steadyScale = 2.5 |
| + } |
| + } |
| + } |
| + |
| + private func resetZoom() { |
| + scale = 1 |
| + steadyScale = 1 |
| + resetPan() |
| + } |
| + |
| + private func resetPan() { |
| + offset = .zero |
| + steadyOffset = .zero |
| + } |
| + |
| private func close() { |
| - withAnimation(.spring(response: 0.3, dampingFraction: 0.9)) { shown = false } |
| + // Unzoom as part of the dismissal, or the fly-back to the bubble |
| + // starts from a transform the thumbnail never had. |
| + withAnimation(.spring(response: 0.3, dampingFraction: 0.9)) { |
| + resetZoom() |
| + shown = false |
| + } |
| // Tear down once the zoom-back has played. |
| DispatchQueue.main.asyncAfter(deadline: .now() + 0.3) { onClose() } |
| } |
diff --git a/ios/LChatMac/MediaLinksDocsSheet.swift b/ios/LChatMac/MediaLinksDocsSheet.swift| index 304d711..3cad895 100644 |
| --- a/ios/LChatMac/MediaLinksDocsSheet.swift |
| +++ b/ios/LChatMac/MediaLinksDocsSheet.swift |
| @@ -6,13 +6,13 @@ import SwiftUI |
| // 2026-08-05-media-links-docs-design.md § Native; iOS twin |
| // MediaLinksDocsSheet.swift, web twin mediaview.js). Same shared extraction |
| // (ThreadMedia over MessageStore) and the same auto-run per-thread history |
| -// walk; presentation is the Mac's fixed-size sheet idiom (Discover's shape). |
| -// Viewing honours the Mac's deliberate anti-QuickLook stance (Lightbox.swift: |
| -// decrypted bytes never touch disk uninvited): photos open an in-sheet |
| -// viewer with ‹ › / arrow-key collection navigation, all in memory; a video |
| -// tile shows its poster there (playing rides "Show in chat", where the feed's |
| -// in-memory player lives); a doc tap saves to ~/Downloads — the feed's own |
| -// file behaviour. |
| +// walk. It lives in its own RESIZABLE window (MediaWindows) rather than a |
| +// sheet — see that file for why. Viewing honours the Mac's deliberate |
| +// anti-QuickLook stance (Lightbox.swift: decrypted bytes never touch disk |
| +// uninvited): photos open an in-window viewer with ‹ › / arrow-key |
| +// collection navigation, all in memory; a video tile shows its poster there |
| +// (playing rides "Show in chat", where the feed's in-memory player lives); a |
| +// doc tap saves to ~/Downloads — the feed's own file behaviour. |
| struct MediaLinksDocsSheet: View { |
| let instance: SavedInstance |
| let thread: ChatThread |
| @@ -20,32 +20,39 @@ struct MediaLinksDocsSheet: View { |
| let model: ChatFeedModel |
| let onJump: (Int64) -> Void |
| - @Environment(\.dismiss) private var dismiss |
| @Environment(\.openURL) private var openURL |
| @State private var tabs = ThreadMedia.Tabs() |
| @State private var loaded = false |
| @State private var tab = "media" |
| @StateObject private var backfill = HistoryBackfill() |
| + // What's IN each photo (Vision, on device) — search matches "dog" or the |
| + // text on a whiteboard. Indexes the thread's stills on open, resumably. |
| + @StateObject private var imageIndex = ImageSearchIndex() |
| // The in-sheet photo viewer's position in the media tab (nil = closed). |
| @State private var viewerIndex: Int? |
| - @State private var savedDocID: String? // brief "Saved to Downloads" state |
| + @State private var query = "" |
| + // Unfurl titles the Links tab has fetched so far, so search matches a |
| + // link by title too (a title only exists once its row was on screen — |
| + // search never triggers fetches of its own). |
| + @State private var linkTitles: [URL: String] = [:] |
| + @State private var savingID: String? // one save at a time (blob id) |
| private var hiddenUids: Set<String> { store.mutedPeople.union(store.blockedPeople) } |
| - private var mediaEntries: [ThreadMedia.Entry] { |
| - ThreadMedia.filtered(tabs.media, hiddenUids: hiddenUids) |
| + // Moderation applies always; the query narrows the open tab (and the |
| + // viewer's collection with it, so ‹ › walk the search results). |
| + private func visible(_ entries: [ThreadMedia.Entry]) -> [ThreadMedia.Entry] { |
| + // `_ = imageIndex.tick` isn't needed: the @StateObject's @Published |
| + // tick re-renders the body, which re-runs this filter. |
| + ThreadMedia.filtered(ThreadMedia.filtered(entries, hiddenUids: hiddenUids), |
| + query: query, titleFor: { linkTitles[$0] }, |
| + imageTextFor: { imageIndex.textFor($0) }) |
| } |
| + private var mediaEntries: [ThreadMedia.Entry] { visible(tabs.media) } |
| + |
| var body: some View { |
| VStack(spacing: 0) { |
| - HStack { |
| - Text("Media, links and docs") |
| - .font(.headline) |
| - Spacer() |
| - Button("Done") { dismiss() } |
| - .keyboardShortcut(.cancelAction) |
| - } |
| - .padding(12) |
| Picker("Kind", selection: $tab) { |
| Text(segLabel("Media", mediaEntries.count)).tag("media") |
| Text(segLabel("Links", ThreadMedia.filtered(tabs.links, hiddenUids: hiddenUids).count)).tag("links") |
| @@ -54,53 +61,138 @@ struct MediaLinksDocsSheet: View { |
| .pickerStyle(.segmented) |
| .labelsHidden() |
| .padding(.horizontal, 12) |
| - if backfill.running { |
| - HStack(spacing: 8) { |
| - Text(backfill.progress ?? "Loading earlier messages…") |
| - .font(.system(size: 12)) |
| - .foregroundStyle(.secondary) |
| - Spacer(minLength: 0) |
| - Button("Stop") { backfill.cancel() } |
| - .buttonStyle(.plain) |
| - .font(.system(size: 12, weight: .semibold)) |
| + .padding(.top, 10) |
| + // Search within the open tab — sender, URL/title, filename |
| + // (ThreadMedia.filtered(query:)); the sidebar search's shape. |
| + HStack(spacing: 6) { |
| + Image(systemName: "magnifyingglass") |
| + .font(.system(size: 11)) |
| + .foregroundStyle(.tertiary) |
| + TextField("Search", text: $query) |
| + .textFieldStyle(.plain) |
| + .font(.system(size: 13)) |
| + .accessibilityLabel("Search media, links and docs") |
| + if !query.isEmpty { |
| + Button { |
| + query = "" |
| + } label: { |
| + Image(systemName: "xmark.circle.fill") |
| + .foregroundStyle(.tertiary) |
| + } |
| + .buttonStyle(.plain) |
| + .accessibilityLabel("Clear search") |
| } |
| - .padding(.horizontal, 12) |
| - .padding(.top, 6) |
| - .accessibilityElement(children: .combine) |
| + } |
| + .padding(.horizontal, 8) |
| + .padding(.vertical, 5) |
| + .background(RoundedRectangle(cornerRadius: 7, style: .continuous) |
| + .fill(.primary.opacity(0.06))) |
| + .padding(.horizontal, 12) |
| + .padding(.top, 8) |
| + if backfill.running { |
| + progressRow(backfill.progress ?? "Loading earlier messages…") { backfill.cancel() } |
| + } |
| + if imageIndex.running { |
| + progressRow(imageIndex.progress ?? "Indexing photos…") { imageIndex.cancel() } |
| } |
| content |
| .padding(.top, 6) |
| } |
| - .frame(width: 640, height: 560) |
| + // The window (MediaWindows) owns the size; the content just fills it, |
| + // with a floor so the grid never collapses. |
| + .frame(minWidth: 480, minHeight: 400) |
| .overlay { |
| if let i = viewerIndex, mediaEntries.indices.contains(i) { |
| MediaViewerOverlay(entries: mediaEntries, index: i, |
| source: { viewerSource($0) }, |
| onStep: { viewerIndex = $0 }, |
| + onSave: { if let att = mediaEntries[i].att { saveToDownloads(att) } }, |
| onClose: { viewerIndex = nil }) |
| } |
| } |
| .task(id: backfill.pageTick) { await reload() } |
| .onAppear { startWalkIfIncomplete() } |
| - .onDisappear { backfill.cancel() } |
| + .onDisappear { |
| + backfill.cancel() |
| + imageIndex.cancel() |
| + } |
| } |
| private func segLabel(_ label: String, _ n: Int) -> String { |
| n > 0 ? "\(label) · \(n.formatted())" : label |
| } |
| + // Inline, occupying layout (never a floating toast): a walk's narration |
| + // plus its Stop — the history walk and the photo indexer share the shape. |
| + private func progressRow(_ text: String, stop: @escaping () -> Void) -> some View { |
| + HStack(spacing: 8) { |
| + // The spinner is the "still working" signal: a count that sits on |
| + // "0 of 5" while a slow photo decrypts otherwise reads as stuck. |
| + ProgressView() |
| + .controlSize(.small) |
| + .scaleEffect(0.7) |
| + .frame(width: 12, height: 12) |
| + Text(text) |
| + .font(.system(size: 12)) |
| + .foregroundStyle(.secondary) |
| + Spacer(minLength: 0) |
| + Button("Stop", action: stop) |
| + .buttonStyle(.plain) |
| + .font(.system(size: 12, weight: .semibold)) |
| + } |
| + .padding(.horizontal, 12) |
| + .padding(.top, 6) |
| + .accessibilityElement(children: .combine) |
| + } |
| + |
| + // Save a photo/video/file to ~/Downloads and reveal it — the Mac feed's |
| + // own file behaviour (a deliberate save, never a silent temp file). |
| + private func saveToDownloads(_ att: Attachment) { |
| + guard savingID == nil else { return } |
| + savingID = att.id |
| + Task { |
| + defer { savingID = nil } |
| + guard let data = await model.blobData(att) else { return } |
| + let name = AttachmentDownload.safeFilename(name: att.name, mime: att.type) |
| + let dir = FileManager.default.urls(for: .downloadsDirectory, in: .userDomainMask).first |
| + ?? FileManager.default.temporaryDirectory |
| + let dest = uniqueDestination(in: dir, filename: name) |
| + guard (try? data.write(to: dest, options: .atomic)) != nil else { return } |
| + NSWorkspace.shared.activateFileViewerSelecting([dest]) |
| + } |
| + } |
| + |
| + // Never overwrite: "name 2.ext", the Finder rule (NativeFeedView's twin). |
| + private func uniqueDestination(in dir: URL, filename: String) -> URL { |
| + let fm = FileManager.default |
| + let name = filename.isEmpty ? "download" : filename |
| + var candidate = dir.appendingPathComponent(name) |
| + guard fm.fileExists(atPath: candidate.path) else { return candidate } |
| + let ext = (name as NSString).pathExtension |
| + let base = (name as NSString).deletingPathExtension |
| + var n = 2 |
| + repeat { |
| + let next = ext.isEmpty ? "\(base) \(n)" : "\(base) \(n).\(ext)" |
| + candidate = dir.appendingPathComponent(next) |
| + n += 1 |
| + } while fm.fileExists(atPath: candidate.path) |
| + return candidate |
| + } |
| + |
| @ViewBuilder private var content: some View { |
| - let entries = ThreadMedia.filtered( |
| - tab == "media" ? tabs.media : tab == "links" ? tabs.links : tabs.docs, |
| - hiddenUids: hiddenUids) |
| + let all = tab == "media" ? tabs.media : tab == "links" ? tabs.links : tabs.docs |
| + let entries = visible(all) |
| if entries.isEmpty { |
| Spacer() |
| - if loaded { |
| + if !loaded { |
| + ProgressView() |
| + } else if !ThreadMedia.filtered(all, hiddenUids: hiddenUids).isEmpty { |
| + Text("Nothing matches.") |
| + .foregroundStyle(.secondary) |
| + } else { |
| Text(tab == "media" ? "No photos or videos shared yet." |
| : tab == "links" ? "No links shared yet." : "No files shared yet.") |
| .foregroundStyle(.secondary) |
| - } else { |
| - ProgressView() |
| } |
| Spacer() |
| } else { |
| @@ -137,6 +229,7 @@ struct MediaLinksDocsSheet: View { |
| .accessibilityLabel("\(att.isVideo ? "Video" : "Photo") from \(e.name.isEmpty ? "someone" : e.name), \(relTime(e.ts / 1000))") |
| .accessibilityAddTraits(.isButton) |
| .contextMenu { |
| + Button("Save to Downloads…") { saveToDownloads(att) } |
| Button("Show in Chat") { onJump(e.sid) } |
| } |
| } |
| @@ -160,14 +253,22 @@ struct MediaLinksDocsSheet: View { |
| private func linkRow(_ e: ThreadMedia.Entry) -> some View { |
| LinkUnfurlRow(e: e, unfurl: { await model.unfurl($0) }, |
| peek: { model.cachedUnfurl($0) }, |
| - open: { openURL($0) }) |
| + open: { openURL($0) }, |
| + onTitle: { url, title in linkTitles[url] = title }) |
| .contextMenu { |
| + if let url = e.url { |
| + Button("Open in Browser") { openURL(url) } |
| + Button("Copy Link") { |
| + NSPasteboard.general.clearContents() |
| + NSPasteboard.general.setString(url.absoluteString, forType: .string) |
| + } |
| + } |
| Button("Show in Chat") { onJump(e.sid) } |
| } |
| } |
| private func docRow(_ e: ThreadMedia.Entry) -> some View { |
| - Button { if let att = e.att { saveDoc(att) } } label: { |
| + Button { if let att = e.att { saveToDownloads(att) } } label: { |
| HStack(spacing: 10) { |
| Text(fileIcon(e.att?.name)) |
| .font(.system(size: 18)) |
| @@ -181,10 +282,13 @@ struct MediaLinksDocsSheet: View { |
| .lineLimit(1) |
| } |
| Spacer(minLength: 0) |
| - if savedDocID == e.att?.id { |
| - Text("Saved to Downloads") |
| - .font(.system(size: 11)) |
| - .foregroundStyle(.secondary) |
| + // The save is a fetch + decrypt; show it's running rather |
| + // than leaving the row inert until Finder pops. |
| + if savingID == e.att?.id { |
| + ProgressView() |
| + .controlSize(.small) |
| + .scaleEffect(0.7) |
| + .frame(width: 12, height: 12) |
| } |
| } |
| .padding(.horizontal, 14) |
| @@ -192,7 +296,11 @@ struct MediaLinksDocsSheet: View { |
| .contentShape(Rectangle()) |
| } |
| .buttonStyle(.plain) |
| + .modifier(RowHover()) |
| .contextMenu { |
| + if let att = e.att { |
| + Button("Save to Downloads…") { saveToDownloads(att) } |
| + } |
| Button("Show in Chat") { onJump(e.sid) } |
| } |
| } |
| @@ -204,32 +312,6 @@ struct MediaLinksDocsSheet: View { |
| return parts.joined(separator: " · ") |
| } |
| - // A doc tap decrypts and saves to ~/Downloads — the Mac feed's file-chip |
| - // behaviour (a deliberate save, never a silent temp file). |
| - private func saveDoc(_ att: Attachment) { |
| - Task { |
| - guard let data = await model.blobData(att) else { return } |
| - let name = AttachmentDownload.safeFilename(name: att.name, mime: att.type) |
| - let downloads = FileManager.default.urls(for: .downloadsDirectory, |
| - in: .userDomainMask).first |
| - guard var dest = downloads?.appendingPathComponent(name) else { return } |
| - // Never overwrite: "name (2).ext", the Finder rule. |
| - var n = 2 |
| - let base = dest.deletingPathExtension().lastPathComponent |
| - let ext = dest.pathExtension |
| - while FileManager.default.fileExists(atPath: dest.path) { |
| - let numbered = ext.isEmpty ? "\(base) (\(n))" : "\(base) (\(n)).\(ext)" |
| - dest = dest.deletingLastPathComponent().appendingPathComponent(numbered) |
| - n += 1 |
| - } |
| - try? data.write(to: dest) |
| - savedDocID = att.id |
| - DispatchQueue.main.asyncAfter(deadline: .now() + 2.5) { |
| - if savedDocID == att.id { savedDocID = nil } |
| - } |
| - } |
| - } |
| - |
| // First load sweeps the whole store; each backfill page then folds ONLY |
| // the newly-filed (strictly older) rows — the web overlay's incremental |
| // onPage fold, not a full re-extract per tick. |
| @@ -253,6 +335,9 @@ struct MediaLinksDocsSheet: View { |
| oldestSID = first.id // page() returns ascending, so first = oldest |
| } |
| loaded = true |
| + // Every photo the thread now holds gets looked at once (on device). |
| + imageIndex.ensure(instance: instance.id, entries: tabs.media, |
| + load: { await model.blobData($0) }) |
| } |
| private func startWalkIfIncomplete() { |
| @@ -316,6 +401,7 @@ private struct LinkUnfurlRow: View { |
| let unfurl: (URL) async -> LinkPreview? |
| let peek: (URL) -> LinkPreview?? |
| let open: (URL) -> Void |
| + let onTitle: (URL, String) -> Void // feeds the sheet's search-by-title memo |
| @State private var title: String? |
| @@ -341,39 +427,79 @@ private struct LinkUnfurlRow: View { |
| .contentShape(Rectangle()) |
| } |
| .buttonStyle(.plain) |
| + .modifier(RowHover()) |
| .onHover { $0 ? NSCursor.pointingHand.push() : NSCursor.pop() } |
| .task(id: e.id) { |
| guard let url = e.url else { return } |
| - if case .some(let hit) = peek(url) { title = hit?.title; return } |
| - title = await unfurl(url)?.title |
| + if case .some(let hit) = peek(url) { title = hit?.title } else { title = await unfurl(url)?.title } |
| + if let title, !title.isEmpty { onTitle(url, title) } |
| } |
| } |
| } |
| -// 🤖 The in-sheet collection viewer: the current photo (or a video's poster) |
| +// Rows highlight under the pointer — the sidebar hover idiom |
| +// (NativeChatView's `.primary.opacity(0.08)` wash), inset a touch so the |
| +// wash doesn't kiss the sheet's edges. |
| +private struct RowHover: ViewModifier { |
| + @State private var hovering = false |
| + func body(content: Content) -> some View { |
| + content |
| + .background(RoundedRectangle(cornerRadius: 8, style: .continuous) |
| + .fill(.primary.opacity(hovering ? 0.07 : 0)) |
| + .padding(.horizontal, 6)) |
| + .onHover { hovering = $0 } |
| + } |
| +} |
| + |
| +// 🤖 The in-window collection viewer: the current photo (or a video's poster) |
| // over a scrim, ‹ › buttons + ←/→ arrow keys stepping through the whole tab, |
| // clamped at the ends (no wrap — the web lightbox's stepIndex rule). All in |
| // memory via BlobImageLoader; Escape (or a scrim click) closes the viewer |
| -// without closing the sheet. |
| +// without closing the window. |
| +// |
| +// Pinch to zoom, drag to pan, double-click to toggle — iOS gets all three |
| +// from Quick Look, so the Mac hand-rolls them to keep the two in step (the |
| +// feed's own lightbox deliberately doesn't zoom; here it's the whole point, |
| +// since this is where you go to read the text in a screenshot). Zoom resets |
| +// when you step to another photo, so every arrival starts fitted. |
| private struct MediaViewerOverlay: View { |
| let entries: [ThreadMedia.Entry] |
| let index: Int |
| let source: (ThreadMedia.Entry) -> AvatarSource? |
| let onStep: (Int) -> Void |
| + let onSave: () -> Void |
| let onClose: () -> Void |
| @State private var image: NSImage? |
| + // Live gesture values plus what they settle to, so a second pinch picks |
| + // up where the last left off (the feed lightbox's steady/live split). |
| + @State private var scale: CGFloat = 1 |
| + @State private var steadyScale: CGFloat = 1 |
| + @State private var offset: CGSize = .zero |
| + @State private var steadyOffset: CGSize = .zero |
| + |
| + private let maxScale: CGFloat = 8 |
| var body: some View { |
| ZStack { |
| Color.black.opacity(0.82) |
| .onTapGesture { onClose() } |
| if let image { |
| + // Fills the (resizable) window, so a screenshot full of small |
| + // text is readable by growing the window — then zooming in. |
| Image(nsImage: image) |
| .resizable() |
| .interpolation(.high) |
| .aspectRatio(contentMode: .fit) |
| - .padding(36) |
| + .scaleEffect(scale) |
| + .offset(offset) |
| + .gesture(magnify.simultaneously(with: pan)) |
| + .onTapGesture(count: 2) { toggleZoom() } |
| + .padding(.horizontal, 12) |
| + .padding(.top, 44) |
| + .padding(.bottom, 12) |
| + .clipped() // a zoomed photo stays inside the viewer |
| + .accessibilityLabel("Photo \((index + 1).formatted()) of \(entries.count.formatted())") |
| } else { |
| ProgressView() |
| .controlSize(.large) |
| @@ -390,8 +516,20 @@ private struct MediaViewerOverlay: View { |
| } |
| .padding(.horizontal, 8) |
| VStack { |
| - HStack { |
| + HStack(spacing: 10) { |
| Spacer() |
| + Button(action: onSave) { |
| + Label("Save", systemImage: "arrow.down.circle") |
| + .font(.system(size: 12, weight: .semibold)) |
| + .foregroundStyle(.white) |
| + .padding(.horizontal, 10) |
| + .padding(.vertical, 5) |
| + .background(.black.opacity(0.45), in: Capsule()) |
| + } |
| + .buttonStyle(.plain) |
| + .keyboardShortcut("s", modifiers: .command) |
| + .accessibilityLabel("Save photo to Downloads") |
| + .arrowCursor() |
| Button { |
| onClose() |
| } label: { |
| @@ -417,10 +555,56 @@ private struct MediaViewerOverlay: View { |
| } |
| .task(id: entries[index].id) { |
| image = nil |
| + resetZoom() |
| guard let src = source(entries[index]) else { return } |
| image = await BlobImageLoader.shared.image(src, maxPixel: nil) |
| } |
| - .accessibilityLabel("Photo \((index + 1).formatted()) of \(entries.count.formatted())") |
| + } |
| + |
| + // Trackpad pinch. Clamped at 1 (fitted) so a photo can't shrink away. |
| + private var magnify: some Gesture { |
| + MagnifyGesture() |
| + .onChanged { value in |
| + scale = min(max(steadyScale * value.magnification, 1), maxScale) |
| + } |
| + .onEnded { _ in |
| + steadyScale = scale |
| + if scale <= 1 { resetPan() } |
| + } |
| + } |
| + |
| + // Drag pans, but only while zoomed — at 1× a drag would just slide the |
| + // fitted photo off its own scrim. |
| + private var pan: some Gesture { |
| + DragGesture() |
| + .onChanged { value in |
| + guard scale > 1 else { return } |
| + offset = CGSize(width: steadyOffset.width + value.translation.width, |
| + height: steadyOffset.height + value.translation.height) |
| + } |
| + .onEnded { _ in steadyOffset = offset } |
| + } |
| + |
| + private func toggleZoom() { |
| + withAnimation(.easeOut(duration: 0.18)) { |
| + if scale > 1 { |
| + resetZoom() |
| + } else { |
| + scale = 2.5 |
| + steadyScale = 2.5 |
| + } |
| + } |
| + } |
| + |
| + private func resetZoom() { |
| + scale = 1 |
| + steadyScale = 1 |
| + resetPan() |
| + } |
| + |
| + private func resetPan() { |
| + offset = .zero |
| + steadyOffset = .zero |
| } |
| private func stepButton(_ symbol: String, _ label: String, enabled: Bool, |
diff --git a/ios/LChatMac/MediaWindows.swift b/ios/LChatMac/MediaWindows.swift| new file mode 100644 |
| index 0000000..84f2574 |
| --- /dev/null |
| +++ b/ios/LChatMac/MediaWindows.swift |
| @@ -0,0 +1,59 @@ |
| +import AppKit |
| +import LChatCore |
| +import SwiftUI |
| + |
| +// 🤖 "Media, links and docs" as a real window, one per chat — the spec's |
| +// "window-pushed view on Mac" (docs/superpowers/specs/ |
| +// 2026-08-05-media-links-docs-design.md § Native). A sheet was the first |
| +// shape and it was wrong: macOS sheets can't be resized, and this surface is |
| +// exactly the one you want to make big — a screenshot full of small text is |
| +// unreadable at any fixed size. A window resizes, zooms, and goes full |
| +// screen, and the collection viewer inside it grows to match. |
| +// |
| +// LChatMacApp deliberately uses `Window` (not `WindowGroup`), so `openWindow` |
| +// isn't available: hand-rolled NSWindow + NSHostingController is the |
| +// sanctioned pattern here, same as ThreadDebugWindows. |
| +@MainActor |
| +enum MediaWindows { |
| + private static var windows: [String: NSWindow] = [:] |
| + private static var closeObservers: [String: NSObjectProtocol] = [:] |
| + |
| + static func show(instance: SavedInstance, thread: ChatThread, store: ThreadStore, |
| + model: ChatFeedModel, onJump: @escaping (Int64) -> Void) { |
| + let key = "\(instance.origin)|\(thread.id)" |
| + if let w = windows[key] { |
| + w.makeKeyAndOrderFront(nil) |
| + return |
| + } |
| + let view = MediaLinksDocsSheet(instance: instance, thread: thread, store: store, |
| + model: model, |
| + onJump: { sid in |
| + close(key) |
| + onJump(sid) |
| + }) |
| + let w = NSWindow(contentViewController: NSHostingController(rootView: view)) |
| + w.title = "Media, links and docs" |
| + w.subtitle = thread.title |
| + w.setContentSize(NSSize(width: 820, height: 660)) |
| + // The dict keeps the window alive; releasedWhenClosed would have |
| + // AppKit deallocate it under us on close. |
| + w.isReleasedWhenClosed = false |
| + windows[key] = w |
| + closeObservers[key] = NotificationCenter.default.addObserver( |
| + forName: NSWindow.willCloseNotification, object: w, queue: .main |
| + ) { _ in |
| + Task { @MainActor in |
| + windows[key] = nil |
| + if let o = closeObservers.removeValue(forKey: key) { |
| + NotificationCenter.default.removeObserver(o) |
| + } |
| + } |
| + } |
| + w.center() |
| + w.makeKeyAndOrderFront(nil) |
| + } |
| + |
| + private static func close(_ key: String) { |
| + windows[key]?.close() |
| + } |
| +} |
diff --git a/ios/LChatMac/NativeChatView.swift b/ios/LChatMac/NativeChatView.swift| index 6574139..4accc78 100644 |
| --- a/ios/LChatMac/NativeChatView.swift |
| +++ b/ios/LChatMac/NativeChatView.swift |
| @@ -268,17 +268,6 @@ struct NativeChatView: View { |
| .sheet(item: $renameSheetThread) { t in |
| RenameGroupSheet(store: store, thread: t) |
| } |
| - .sheet(item: $mediaSheetThread) { t in |
| - MediaLinksDocsSheet(instance: instance, thread: t, store: store, |
| - model: feedModel(t), |
| - onJump: { sid in |
| - mediaSheetThread = nil |
| - // Land the (already-selected or not) feed |
| - // on the message — the search hit's path. |
| - selectedID = t.id |
| - app.nativeJump = .init(thread: t.id, message: sid) |
| - }) |
| - } |
| .sheet(item: $appsSheetThread) { t in |
| GroupAppsSheet(model: feedModel(t)) |
| } |
| @@ -792,8 +781,6 @@ struct NativeChatView: View { |
| @State private var membersSheetThread: ChatThread? |
| @State private var renameSheetThread: ChatThread? |
| - // The per-thread shared-content inventory (spec: media-links-docs). |
| - @State private var mediaSheetThread: ChatThread? |
| // A DM header's profile card (rename them for yourself there — #577). |
| @State private var headerProfile: HeaderProfileTarget? |
| struct HeaderProfileTarget: Identifiable { let id: String } |
| @@ -818,7 +805,18 @@ struct NativeChatView: View { |
| // Media, Links & Docs — DMs and groups alike (NOT in the group |
| // block); app threads have no shared-content feed, so it hides there. |
| if !t.isApp { |
| - Button("Media, Links & Docs…") { mediaSheetThread = t } |
| + // Its own window (resizable — a screenshot full of small text |
| + // needs to be made big), not a sheet: MediaWindows. |
| + Button("Media, Links & Docs…") { |
| + MediaWindows.show(instance: instance, thread: t, store: store, |
| + model: feedModel(t), |
| + onJump: { sid in |
| + // Land the feed on the message — the |
| + // search hit's path. |
| + selectedID = t.id |
| + app.nativeJump = .init(thread: t.id, message: sid) |
| + }) |
| + } |
| Divider() |
| } |
| Button(t.pinned ? "Unpin" : "Pin") { store.setPinned(t, !t.pinned) } |
diff --git a/test/threadmedia.test.mjs b/test/threadmedia.test.mjs| index 952fa0c..5262260 100644 |
| --- a/test/threadmedia.test.mjs |
| +++ b/test/threadmedia.test.mjs |
| @@ -1,7 +1,7 @@ |
| // Media, Links & Docs: the pure extractor + in-memory index |
| // (web/static/threadmedia.js). Node-run: node test/threadmedia.test.mjs |
| import assert from "node:assert/strict"; |
| -import { extractEntries, bucketByMonth, monthLabel, ThreadMediaIndex } from "../web/static/threadmedia.js"; |
| +import { extractEntries, bucketByMonth, monthLabel, ThreadMediaIndex, filterEntries } from "../web/static/threadmedia.js"; |
| let failures = 0; |
| async function test(name, fn) { |
| @@ -100,4 +100,24 @@ await test("decrypt failure on one row doesn't poison the rest", async () => { |
| assert.equal(idx.tab("t1", "media", { hiddenUids: new Set() }).length, 1); |
| }); |
| +await test("filterEntries matches sender, URL, fetched unfurl title, and filename — case-insensitively", () => { |
| + const entries = [ |
| + { sid: 1, name: "Ada", url: "https://example.com/Report" }, |
| + { sid: 2, name: "Bob", url: "https://news.example/x" }, |
| + { sid: 3, name: "Cy", att: { id: "d", name: "Budget-2026.xlsx" } }, |
| + { sid: 4, name: "Ada", att: { id: "p", type: "image/png" } }, |
| + ]; |
| + const titles = { "https://news.example/x": "Rain expected Friday" }; |
| + const titleFor = (u) => titles[u] || ""; |
| + const ids = (q) => filterEntries(entries, q, titleFor).map((e) => e.sid); |
| + assert.deepEqual(ids(""), [1, 2, 3, 4], "blank query = everything"); |
| + assert.deepEqual(ids(" "), [1, 2, 3, 4]); |
| + assert.deepEqual(ids("ada"), [1, 4], "sender name, any tab"); |
| + assert.deepEqual(ids("REPORT"), [1], "URL substring, case-insensitive"); |
| + assert.deepEqual(ids("friday"), [2], "an already-fetched unfurl title"); |
| + assert.deepEqual(ids("budget"), [3], "doc filename"); |
| + assert.deepEqual(ids("zzz"), []); |
| + assert.deepEqual(filterEntries(entries, "friday").map((e) => e.sid), [], "no titleFor → titles don't match"); |
| +}); |
| + |
| process.exit(failures ? 1 : 0); |
diff --git a/tools/dev b/tools/dev| index 0fa685a..fb2aee8 100755 |
| --- a/tools/dev |
| +++ b/tools/dev |
| @@ -7,6 +7,9 @@ |
| // ./tools/dev run mac Launch the Mac app |
| // ./tools/dev alt [slug] Build + launch a side-by-side dev alt of the Mac app |
| // ./tools/dev run ios Run the iOS app on simulator (see instructions) |
| +// ./tools/dev install ios [id] Build + install + launch on a real iPhone (network, |
| +// no cable); id defaults to the first paired device, |
| +// or LCHAT_IOS_DEVICE |
| // ./tools/dev up [flags] Start the local dev server (persistent, reusable config) |
| // + the webhook-ingest sidecar. Flags after the command |
| // pass straight to tools/lchat-dev-up.mjs: --clean, |
| @@ -51,6 +54,36 @@ function needsXcodegen() { |
| } |
| } |
| +// The first physical iPhone devicectl reports as usable — what `install ios` |
| +// targets when no id is given. Simulators and the header row are skipped. |
| +function firstPairediPhone() { |
| + try { |
| + const out = execFileSync("xcrun", ["devicectl", "list", "devices"], { encoding: "utf8" }); |
| + for (const line of out.split("\n")) { |
| + if (!/(available|connected)/i.test(line)) continue; |
| + if (!/physical/i.test(line) || /simulated/i.test(line)) continue; |
| + const id = line.match(/([A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12})/i); |
| + if (id) return id[1]; |
| + } |
| + } catch { |
| + // devicectl unavailable / no devices — the caller prints the hint. |
| + } |
| + return null; |
| +} |
| + |
| +// The app's bundle id, read from project.yml's one knob (LCHAT_APP_BUNDLE_ID) |
| +// so an overridden id still launches the thing we just installed. |
| +function bundleIdFromProject() { |
| + try { |
| + const yml = readFileSync(path.join(IOS_DIR, "project.yml"), "utf8"); |
| + const hit = yml.match(/LCHAT_APP_BUNDLE_ID:\s*(\S+)/); |
| + if (hit) return hit[1]; |
| + } catch { |
| + // fall through to the shipped default |
| + } |
| + return "com.elevenmessenger.messenger"; |
| +} |
| + |
| // Check if build needs to be redone (source files changed since app was built) |
| function needsBuild(appPath) { |
| if (!existsSync(appPath)) return true; |
| @@ -137,6 +170,58 @@ const commands = { |
| console.log("✓ iOS app built successfully"); |
| }, |
| + // 🤖 Build + install + launch on a REAL iPhone, over the network tunnel |
| + // devicectl opens (no cable: the phone just has to be unlocked, trusted and |
| + // on the same Wi-Fi). Encoded here rather than living in an operator's |
| + // private notes, so any machine can put a dev build on a device. |
| + // ./tools/dev install ios [device-id] (or LCHAT_IOS_DEVICE=<id>) |
| + // Signing is Automatic + -allowProvisioningUpdates: a new device must |
| + // already be registered with the team before a profile can be minted. |
| + // If xcodebuild says "iOS <v> is not installed", point DEVELOPER_DIR at an |
| + // Xcode that carries that platform (see docs/native-chat.md). |
| + "install ios": () => { |
| + if (needsXcodegen()) { |
| + console.log("Generating Xcode project..."); |
| + run("xcodegen", ["generate"], { cwd: IOS_DIR }); |
| + } |
| + const device = cmdArgs[0] || process.env.LCHAT_IOS_DEVICE || firstPairediPhone(); |
| + if (!device) { |
| + console.error("No paired iPhone found. Unlock it, trust this Mac, same Wi-Fi;"); |
| + console.error("`xcrun devicectl list devices` should show it as available."); |
| + process.exit(1); |
| + } |
| + console.log(`Building LChat for device ${device}...`); |
| + const derived = path.join(IOS_DIR, "build"); |
| + run("xcodebuild", [ |
| + "-project", path.join(IOS_DIR, "LChat.xcodeproj"), |
| + "-scheme", "LChat", |
| + "-destination", "generic/platform=iOS", |
| + "-derivedDataPath", derived, |
| + "-allowProvisioningUpdates", |
| + "CODE_SIGN_STYLE=Automatic", |
| + "build", |
| + ], { cwd: IOS_DIR }); |
| + const app = path.join(derived, "Build/Products/Debug-iphoneos/LChat.app"); |
| + // The first reach for a sleeping tunnel often comes back "Connection reset |
| + // by peer"; the immediate retry lands. Only a second failure is real. |
| + console.log("Installing..."); |
| + const install = () => spawnSync("xcrun", |
| + ["devicectl", "device", "install", "app", "--device", device, app], |
| + { cwd: REPO_ROOT, stdio: "inherit" }); |
| + if (install().status !== 0) { |
| + console.log("Device tunnel didn't answer — retrying once..."); |
| + if (install().status !== 0) { |
| + console.error("Install failed. Unlock the phone and make sure it's on this Wi-Fi;"); |
| + console.error("opening Xcode → Window → Devices and Simulators (⇧⌘2) kicks the tunnel."); |
| + process.exit(1); |
| + } |
| + } |
| + console.log("Launching..."); |
| + run("xcrun", ["devicectl", "device", "process", "launch", "--device", device, |
| + bundleIdFromProject()]); |
| + console.log("✓ Installed and launched on the device"); |
| + }, |
| + |
| "up": async () => { |
| console.log("Starting dev server (lchat-dev-up)..."); |
| // Pass flags through (--clean, --onboarding, --no-hooks, --port N, …). |
| @@ -616,6 +701,8 @@ if (!cmd || wantsHelp) { |
| console.log(" (orange icon; --icon live = green, the"); |
| console.log(" convention for an alt aimed at prod home)"); |
| console.log(" ./tools/dev run ios Run iOS app on simulator"); |
| + console.log(" ./tools/dev install ios [id] Build + install + launch on a real iPhone"); |
| + console.log(" (over the network; LCHAT_IOS_DEVICE also sets it)"); |
| console.log(" ./tools/dev up [flags] Start local dev server (flags go to"); |
| console.log(" lchat-dev-up: --clean, --onboarding,"); |
| console.log(" --no-hooks, --port N, --help)"); |
diff --git a/web/static/mediaview.js b/web/static/mediaview.js| index f19ddf0..ffa2015 100644 |
| --- a/web/static/mediaview.js |
| +++ b/web/static/mediaview.js |
| @@ -3,7 +3,7 @@ |
| // thread with Media | Links | Docs tabs, fed by threadmedia.js over the local |
| // store, completed by an auto-run resumable backfill (marks shared with |
| // search — either feature finishing the walk finishes it for both). |
| -import { ThreadMediaIndex, bucketByMonth } from "/static/threadmedia.js"; |
| +import { ThreadMediaIndex, bucketByMonth, filterEntries } from "/static/threadmedia.js"; |
| import { allMessages as storeAllMessages, getBackfillMark as storeBackfillMark, runBackfill as storeRunBackfill } from "/static/store.js"; |
| import { decrypt } from "/static/crypto.js"; |
| import { threadKeys, threadName } from "/static/shellstate.js"; |
| @@ -56,7 +56,13 @@ const TABS = [ |
| { id: "docs", label: "Docs", render: renderDocsTab }, |
| ]; |
| -let open = null; // { tid, root, walk: AbortController|null, tab } — one overlay at a time |
| +let open = null; // { tid, root, walk: AbortController|null, tab, query } — one overlay at a time |
| + |
| +// Unfurl titles the Links tab has already fetched, so search can match a |
| +// link by its title too (a title only exists once its row has scrolled into |
| +// view — search never triggers fetches of its own). Session-lifetime memo, |
| +// like the shell's own unfurl cache. |
| +const linkTitles = new Map(); |
| export async function openMediaView(tid) { |
| closeMediaView(); |
| @@ -67,7 +73,7 @@ export async function openMediaView(tid) { |
| root.setAttribute("role", "dialog"); |
| root.setAttribute("aria-modal", "false"); // the chat stays beneath, inert to pointer only |
| root.setAttribute("aria-label", `Media, links and docs — ${threadName[tid] || "Chat"}`); |
| - open = { tid, root, walk: null, tab: "media", returnFocus: document.activeElement }; |
| + open = { tid, root, walk: null, tab: "media", query: "", returnFocus: document.activeElement }; |
| const head = document.createElement("div"); |
| head.className = "media-head"; |
| @@ -85,6 +91,20 @@ export async function openMediaView(tid) { |
| seg.className = "media-tabs"; |
| seg.setAttribute("role", "tablist"); |
| seg.setAttribute("aria-label", "Kind"); |
| + // Search within the open tab — sender, URL/title, filename (filterEntries). |
| + // Filters as you type; the field survives repaints (it's built once, here). |
| + const search = document.createElement("div"); |
| + search.className = "media-search"; |
| + const searchInput = document.createElement("input"); |
| + searchInput.type = "search"; |
| + searchInput.placeholder = "Search"; |
| + searchInput.setAttribute("aria-label", "Search media, links and docs"); |
| + searchInput.autocomplete = "off"; |
| + searchInput.oninput = () => { |
| + open.query = searchInput.value; |
| + paintBody(); |
| + }; |
| + search.appendChild(searchInput); |
| const body = document.createElement("div"); |
| body.className = "media-body"; |
| // ONE polite live region for the walk's narration; sits in layout (never a |
| @@ -100,7 +120,7 @@ export async function openMediaView(tid) { |
| stop.onclick = () => open && open.walk && open.walk.abort(); |
| progress.append(progressText, stop); |
| - root.append(head, seg, progress, body); |
| + root.append(head, seg, search, progress, body); |
| host.appendChild(root); |
| document.addEventListener("keydown", onOverlayKey); |
| back.focus(); // focus lands in the overlay; Escape/back returns it |
| @@ -134,11 +154,14 @@ export async function openMediaView(tid) { |
| body.replaceChildren(); |
| body.setAttribute("role", "tabpanel"); |
| body.setAttribute("aria-labelledby", `media-tab-${open.tab}`); |
| - const entries = mediaIdx.tab(tid, open.tab, { hiddenUids: hiddenUids() }); |
| + const all = mediaIdx.tab(tid, open.tab, { hiddenUids: hiddenUids() }); |
| + const entries = filterEntries(all, open.query, (u) => linkTitles.get(u)); |
| if (!entries.length) { |
| const empty = document.createElement("div"); |
| empty.className = "media-empty"; |
| - empty.textContent = { media: "No photos or videos shared yet.", links: "No links shared yet.", docs: "No files shared yet." }[open.tab]; |
| + empty.textContent = all.length |
| + ? "Nothing matches." |
| + : { media: "No photos or videos shared yet.", links: "No links shared yet.", docs: "No files shared yet." }[open.tab]; |
| body.appendChild(empty); |
| return; |
| } |
| @@ -220,6 +243,15 @@ function onOverlayKey(e) { |
| // the top layer also closes the media overlay underneath it. |
| if (e.key === "Escape" && open && !document.querySelector(".lightbox, .msg-sheet")) { |
| e.preventDefault(); |
| + // Escape in a non-empty search clears the search first (the field's own |
| + // convention); a second Escape closes the overlay. |
| + const input = open.root.querySelector(".media-search input"); |
| + if (input && document.activeElement === input && input.value) { |
| + input.value = ""; |
| + open.query = ""; |
| + open.repaint(); |
| + return; |
| + } |
| closeMediaView(); |
| } |
| } |
| @@ -400,6 +432,7 @@ function renderLinksTab(body, entries, tid) { |
| observeTile(row, async () => { |
| const card = await deps.unfurl(e.url); |
| if (!card || !card.ok || !card.title) return; |
| + linkTitles.set(e.url, card.title); |
| const t = document.createElement("span"); |
| t.className = "media-link-title"; |
| t.textContent = card.title; |
diff --git a/web/static/style.css b/web/static/style.css| index 25b8ad6..06a2820 100644 |
| --- a/web/static/style.css |
| +++ b/web/static/style.css |
| @@ -3000,6 +3000,15 @@ body.command-host .cmd-panel-body { max-height: calc(100vh - 40px); } |
| .media-head { display: flex; align-items: center; gap: 8px; padding: 10px 12px; } |
| .media-title { font-weight: 600; } |
| .media-tabs { display: flex; gap: 6px; padding: 0 12px 10px; } |
| +/* Search within the open tab — the sidebar's .chat-search field, one row down. */ |
| +.media-search { display: flex; padding: 0 12px 8px; } |
| +.media-search input { |
| + flex: 1; min-width: 0; box-sizing: border-box; font: inherit; font-size: 14px; |
| + padding: 8px 12px; border: 1px solid transparent; border-radius: 12px; |
| + background: var(--bubble-them); color: var(--ink); -webkit-appearance: none; appearance: none; |
| +} |
| +.media-search input::placeholder { color: var(--muted); } |
| +.media-search input:focus { background: var(--panel); } |
| .media-tab { |
| padding: 6px 12px; |
| border-radius: 999px; |
diff --git a/web/static/threadmedia.js b/web/static/threadmedia.js| index dbecc2d..e310f2b 100644 |
| --- a/web/static/threadmedia.js |
| +++ b/web/static/threadmedia.js |
| @@ -112,3 +112,22 @@ export class ThreadMediaIndex { |
| this.threads.delete(tid); |
| } |
| } |
| + |
| +// Search within a tab: a case-insensitive substring over what each entry |
| +// actually shows — the sender's name (every tab), the URL and its unfurl |
| +// title (links; `titleFor(url)` returns the title if it's been fetched, else |
| +// ""), and the filename (docs, and media that carry one). Empty/blank query = |
| +// everything. Pure, so the three UIs share one definition of "matches". |
| +export function filterEntries(entries, query, titleFor = () => "") { |
| + const q = (query || "").trim().toLowerCase(); |
| + if (!q) return entries; |
| + return entries.filter((e) => { |
| + if ((e.name || "").toLowerCase().includes(q)) return true; |
| + if (e.url) { |
| + if (e.url.toLowerCase().includes(q)) return true; |
| + if ((titleFor(e.url) || "").toLowerCase().includes(q)) return true; |
| + } |
| + if (e.att && (e.att.name || "").toLowerCase().includes(q)) return true; |
| + return false; |
| + }); |
| +} |