| 1 | # Calls native apps |
| 2 | |
| 3 | The SwiftUI Mac and iPhone apps use the same Calls server and shared Go core. |
| 4 | |
| 5 | ## Build and run |
| 6 | |
| 7 | Requires macOS 14+, Xcode, Go 1.26 and network access for pinned dependencies. |
| 8 | |
| 9 | ```sh |
| 10 | make native-mac |
| 11 | open native/build/Calls.app |
| 12 | ``` |
| 13 | |
| 14 | Signed out, the home screen names the Calls server — `calls.oficina.works` |
| 15 | unless you have chosen another — and offers one button, **Sign in**. It opens |
| 16 | your default browser for the existing Home/Calls sign-in on that server. Open |
| 17 | the emailed link normally, compare the code in the browser with the app, and |
| 18 | confirm **Sign in to Calls app**. The app continues automatically. There is no |
| 19 | address to type. |
| 20 | |
| 21 | **Join a call as a guest**, under it, opens a screen with one field for the |
| 22 | invite link someone sent you (`/j/…` or `/m/…`, on any server), a **Paste** |
| 23 | control, **Join** and **Cancel**. The link is checked before anything opens, |
| 24 | and a problem is said beside the field in plain words. Join runs that meeting's |
| 25 | own browser sign-in — the pending-join for someone who is not a member — on the |
| 26 | server the link names. The clipboard is read only through Paste, so iPhone |
| 27 | never asks whether Calls may paste, and only a Calls link is taken from it. |
| 28 | |
| 29 | **The server is a setting.** On a Mac it is **Calls server** in **Calls → |
| 30 | Settings…**, after the mini-window settings; on iPhone the gear on the home |
| 31 | screen (or **Change** beside the server line) opens a small settings sheet with |
| 32 | the same field. The address must be https — plain http is accepted only for |
| 33 | `localhost`, `127.0.0.1` and `::1`, the rule callscore's transport enforces — with |
| 34 | a host and nothing after it (no path, query or name and password); whitespace |
| 35 | and trailing slashes are trimmed and a bare host is taken to be https. **Reset |
| 36 | to calls.oficina.works** puts the default back. It is kept in UserDefaults (the |
| 37 | default is kept as no value at all, so a later change of default reaches |
| 38 | everyone who never chose). Changing it while signed in asks first, then signs |
| 39 | out of the old server exactly as **Sign out** does — the Keychain copy |
| 40 | forgotten, the old server asked to revoke the session — and it cannot be |
| 41 | changed during a call. `-CallsServer <origin>` on the command line overrides it |
| 42 | for one run without saving it, which is how the iPhone UI tests point the app at |
| 43 | their own server. |
| 44 | |
| 45 | **Links.** A meeting link — `oficina-calls://join?url=…` from the browser, or |
| 46 | one pasted into the signed-in home's field — never changes the server setting. |
| 47 | On the server you are signed in to, it opens with the session the app holds if |
| 48 | that session may join the meeting (`OpenLink` asks the meeting's `/info`); if it |
| 49 | may not — the app is signed in as somebody else — the home says so and offers a |
| 50 | sign-in for that meeting only. A link from a signed-in browser's **Join in the |
| 51 | app** also carries an `offer` (2026-10-02): the app takes it up at once, opens |
| 52 | no browser, and the page the person came from asks them to confirm the app's |
| 53 | code, handing over their own session — member or guest — for that meeting. |
| 54 | Otherwise a link arriving through `oficina-calls://` is offered, not |
| 55 | started: the home names the server the meeting is on with a **Sign in** button |
| 56 | (and **Not now**), so a browser never opens for a server the person has not seen |
| 57 | named. One pasted into the field is the person's own act, so **Open link** goes |
| 58 | straight to that sign-in. A link on another server |
| 59 | signs in there **for that meeting only**, exactly as a guest does: that session |
| 60 | is never kept, starts no meetings, and leaving the meeting (or Sign out) drops |
| 61 | it — asking that server to revoke it — and comes back to the server's own home, |
| 62 | signed in if a member's session is kept, signed out if not. The kept sign-in |
| 63 | for the configured server is not replaced by a visit. |
| 64 | |
| 65 | Once signed in — however sign-in began — the app is home: **Start meeting**, |
| 66 | **Join meeting** when there is a meeting in hand (the link you signed in from, |
| 67 | or the call you just left), a field to open a link someone sent you, and |
| 68 | **Sign out**. A link on the instance you are signed in to opens with the session |
| 69 | the app already holds, pasted or arriving as `oficina-calls://join?url=…`, and |
| 70 | joins straight away; a link on another instance is that instance's sign-in, as |
| 71 | before. Leaving a call comes back home. |
| 72 | |
| 73 | The session is the person's whole session whether the browser landed on a link |
| 74 | or on the meeting list — the link only decides where it lands. Who may start a |
| 75 | meeting is `requireMember`'s decision on the server, and a refusal is shown as |
| 76 | the server gives it. Starting goes through `POST /native/meetings/start` — the |
| 77 | same `store.createMeeting`, behind the same `requireMember`, as the web list's |
| 78 | one-click button — and answers only the meeting's identifier. The invitation is |
| 79 | not part of the answer: a link is a credential, and the room's own **Share** is |
| 80 | where one is minted. |
| 81 | |
| 82 | **The sign-in is kept.** A member's session goes into the Keychain — a generic |
| 83 | password, this device only, readable while unlocked — and the next launch |
| 84 | checks it with the instance before offering anything ("Checking your |
| 85 | sign-in"): one `GET /settings`, member-only, read by status alone. A member's |
| 86 | page means signed in; the sign-in redirect means the session has ended (thirty |
| 87 | days, or revoked), so it is forgotten and the home says so and points at |
| 88 | **Sign in**; anything else — |
| 89 | no network, a 5xx — keeps it, uses it, and offers **Check sign-in again**. Only |
| 90 | a member's session on the configured server is kept: a guest's, an invited |
| 91 | non-member's, or one on another server is good for the meeting it came in |
| 92 | through and is signed in again from the next link. A session kept for a server |
| 93 | that is no longer the configured one (the setting changed outside the app) is |
| 94 | signed out at launch. A sign-in that fails — no network, a server that does not |
| 95 | answer — says what happened and to choose **Sign in** (or **Join a call as a |
| 96 | guest**) to try again. |
| 97 | **Sign out** forgets the Keychain copy at once and asks the instance to revoke |
| 98 | the session with the web's own `POST /signout`; if that cannot be reached, the |
| 99 | session lapses at its thirty days. |
| 100 | |
| 101 | On a Mac, the this-device-only keychain needs a keychain-access-groups |
| 102 | entitlement that the ad hoc signed `make native-mac` build does not carry. |
| 103 | There the item falls back to the login keychain, protected by the login |
| 104 | password and the app's code signature; because an ad hoc signature changes on |
| 105 | every rebuild, macOS may ask once per build whether Calls may read it. |
| 106 | |
| 107 | Scheduling a meeting for later, ending one, and replacing invitations stay on |
| 108 | the web for this milestone. |
| 109 | |
| 110 | **Devices** in the call opens the camera, microphone and speaker panel. The |
| 111 | audio readout is behind its ⓘ (**Audio details**), with the screen-share line |
| 112 | while sharing; the panel itself shows one warning line only when something is |
| 113 | wrong (a stretched or broken-up signal, echo cancellation off, noise reduction |
| 114 | bypassed, processing refused — `AudioWarning`). |
| 115 | |
| 116 | **Sound is measured, not inferred** (spec §15.20). Since 0.3.7 a tile shows it |
| 117 | as the speaking light on the name — since 0.4.3 the warm rim (`SpeakingGlow` |
| 118 | and `WarmRim`: -42 to -12 dBFS, lit once speech has kept up 0.15 s, 50 ms |
| 119 | attack, 300 ms release, a steady 0.6 under Reduce Motion; see DESIGN.md) — in |
| 120 | place of the white ring and the meter. On 2026-09-25 Tony, in a |
| 121 | browser, could not hear Paul on the Mac app, and Paul's panel looked fine |
| 122 | because nothing on it measured sound. Now: |
| 123 | |
| 124 | - **Your tile's rim** and the panel's **Microphone level** meter what is sent — the |
| 125 | capture hook's level after noise reduction (`CaptureLevel`, read at 10 Hz). |
| 126 | Unmuted and capturing, with nothing reaching the hook above -90 dBFS |
| 127 | *before* the suppressor for five seconds, the tile says "Your microphone |
| 128 | isn't picking anything up", and its ⓘ names the device and input being sent |
| 129 | ("Scarlett 18i8 USB, input 1"). Only once the hook has had buffers while a |
| 130 | connection was up, and only while one still is (`MicSample.sending`): |
| 131 | alone in a call WebRTC never opens the microphone, and 0.4.6 told Paul, |
| 132 | alone, that his was dead (2026-10-03). |
| 133 | - **Everyone else's tile** meters what arrives from them: a renderer on their |
| 134 | remote audio track (`RemoteAudioTap`), which sees each decoded frame before |
| 135 | the output volume, read at 5 Hz. Muted, the tile shows the Muted badge and no |
| 136 | glow. Reporting themselves unmuted, connected, with audio bytes still rising |
| 137 | (inbound-rtp, one `statistics(for:)` per receiver a second) and the decoded |
| 138 | peak under -85 dBFS for five seconds, it says "No sound from Bea" instead, |
| 139 | and the app reports those people as `noSoundFrom` through the bridge. A thin |
| 140 | white ring marks whoever is speaking (with hysteresis; no fade under Reduce |
| 141 | Motion), and the mini window's tab shows a waveform while anyone is. |
| 142 | - **Their report about you** shows on their tile: "Tony's browser is blocking |
| 143 | sound — they need to click to hear you", or, while you are unmuted, "Tony |
| 144 | isn't getting any sound from you". Each line is in the tile's accessibility |
| 145 | label and announced when it appears. The app itself is never blocked from |
| 146 | playing sound, so it never reports `audioBlocked`. |
| 147 | - **Who can hear you yet** (spec §15.24): the app reports `hearing`, the people |
| 148 | whose audio path is established here (connected, packets in and decoded |
| 149 | frames at the tap, or connected and muted), read five times a second until |
| 150 | it is. From the moment someone joins, their tile says "Samuel is connecting |
| 151 | and can't hear you yet" — past a one-second grace, with dots that take turns |
| 152 | (steady under Reduce Motion) — until their report lists you, their own |
| 153 | blocked or no-sound report replacing it, and never past twenty seconds |
| 154 | (`ConnectingWatch.swift`). The mini window and its tab show a small ear for |
| 155 | anyone it is not showing. |
| 156 | |
| 157 | The rules (`AudioLevelRules.swift`) are held by `AudioLevelTests` on an |
| 158 | injected clock; `TestNativeMacCallsBrowser` checks that the browser measures |
| 159 | energy, not only bytes, in what the app sends, and that the app's receive meter |
| 160 | hears the browser. |
| 161 | |
| 162 | The camera is a picker on both platforms. Switching rests on the capturer |
| 163 | feeding the same `RTCVideoSource`, so the `RTCVideoTrack` every peer |
| 164 | connection already holds does not change and nothing renegotiates — the |
| 165 | native counterpart of the browser's `replaceTrack`. Switches are serialised; |
| 166 | the last pick wins. |
| 167 | |
| 168 | The camera is remembered and watched (`CameraDirector`, `ManagedCamera`, held |
| 169 | by `CameraRecoveryTests`), because of an ATEM Mini that dropped off USB mid-call |
| 170 | in 0.3.1 and left the self tile on "Waiting for video": |
| 171 | |
| 172 | - **Remembered.** A camera chosen in the panel is kept in UserDefaults by its |
| 173 | `uniqueID`, with its name for while it is missing. A call starts on it if it |
| 174 | is connected, otherwise on the built-in camera (the front one on iPhone), |
| 175 | otherwise on whatever there is, and the panel says so ("ATEM Mini isn't |
| 176 | connected; using FaceTime HD Camera"). |
| 177 | - **Followed.** Cameras connecting and disconnecting are observed — |
| 178 | AVFoundation's connect and disconnect notifications and, on Mac, the |
| 179 | discovery session's device list by KVO — with a look every three seconds as |
| 180 | well while there is no working camera. The camera in use disconnecting falls |
| 181 | back to another at once ("ATEM Mini disconnected; using FaceTime HD Camera"); |
| 182 | the remembered camera coming back, or appearing later, is switched back to. |
| 183 | - **Watched.** Frames are counted on their way to the video source. No first |
| 184 | frame within three seconds of opening, or none for three seconds after they |
| 185 | had been arriving, restarts capture on the same device; still nothing tries |
| 186 | the device's next format, then another camera. With nothing to fall back to, |
| 187 | the camera is named as having no signal and kept, and a frame arriving later |
| 188 | is noticed. A camera that fell silent is returned to only when it reconnects. |
| 189 | - **Formats.** Nearest 640 wide, then nearest 30 frames a second without going |
| 190 | over, at a rate the format really supports: 30 where a range covers it, or |
| 191 | the format's own rate (an ATEM's 1080p formats each run at exactly one of |
| 192 | 23.98, 24, 25, 29.97, 30, 50, 59.94 or 60), set on the device exactly when it |
| 193 | is not a whole number. An open that has not finished in six seconds counts as |
| 194 | refused. |
| 195 | - **Restarted** after a capture session runtime error, an interruption ending, |
| 196 | or the Mac waking, at most once every five seconds. |
| 197 | - **Said.** Your own tile says what the camera is doing instead of an endless |
| 198 | "Waiting for video": "Starting camera…", "No signal from ATEM Mini — check |
| 199 | its input", "Camera disconnected", "No camera found". The panel's Camera row |
| 200 | is always there in a call — "No camera found" with **Try again** when there |
| 201 | is none, and the picker with a plain-words line when the camera is not |
| 202 | working. |
| 203 | |
| 204 | The native/browser gate drives the same code with synthetic cameras, and stalls |
| 205 | one mid-call: the watchdog has to restart it and the browser has to decode the |
| 206 | picture again on the same connection. |
| 207 | |
| 208 | The microphone is a picker on both platforms, and on Mac so is the speaker. |
| 209 | On iPhone it is an audio session route. On Mac it is a CoreAudio device that |
| 210 | the call's AVAudioEngine (LiveKit's engine module) is put on; switching mid-call |
| 211 | rebuilds the engine without renegotiating. A choice there moves this call and |
| 212 | leaves the Mac's system setting alone. Before the move to LiveKit's build the |
| 213 | Mac could only report what macOS had chosen: that build's macOS slice had no |
| 214 | audio device API at all. |
| 215 | |
| 216 | The Mac's choices follow these rules (`AudioRoutes`, held by |
| 217 | `AudioRoutesTests`): |
| 218 | |
| 219 | - **Remembered** across calls and launches, in UserDefaults: the device's |
| 220 | CoreAudio UID, and its name for while it is missing. A device back under a |
| 221 | new UID (a USB interface on another port) is found by that name when exactly |
| 222 | one connected device has it. Calls 0.3.1 forgot them at the end of every |
| 223 | call. The panel says "Calls remembers these for next time." |
| 224 | - **System default** is the first entry of each picker, where someone who has |
| 225 | never chosen starts, and a choice like any other (remembered when picked). It |
| 226 | follows macOS: a new default device, or the device changing its sample rate, |
| 227 | is followed by the module itself, which listens for both and rebuilds its |
| 228 | engine. |
| 229 | - An explicit choice keeps its device; a change of system default does not |
| 230 | move it. |
| 231 | - A chosen device that is not connected — at join or mid-call — puts the call on |
| 232 | the system default for now, and the panel says so ("Scarlett 18i8 isn't |
| 233 | connected; using MacBook Pro Microphone"). "Connected" means CoreAudio's own |
| 234 | device list, never the engine module's. The note never names the missing |
| 235 | device as the one in use. The choice is kept, and the call goes back to the |
| 236 | device when it reappears; CoreAudio's device list and default devices are |
| 237 | listened to for that, as well as the module's own report. No poller. |
| 238 | - A device the engine will not start on (the module gives up without saying |
| 239 | so; the app notices an engine that is not running two seconds after a start, |
| 240 | with no stop since) is set aside: "Calls couldn't open Scarlett 18i8 USB; |
| 241 | using MacBook Pro Microphone", and the call goes back to the default. Choosing |
| 242 | it again, or anything being plugged in or taken out, tries it again. |
| 243 | - Everything above is logged, with device lists, each engine start and the |
| 244 | device its unit is actually on: |
| 245 | `log show --last 1h --predicate 'subsystem == "net.amadan.oficina.calls" && category == "audio"' --info`. |
| 246 | - Nothing changes Sound settings. Mute is not remembered: every call starts |
| 247 | unmuted. |
| 248 | - **Input**, for a microphone with more than one input (an audio interface): |
| 249 | a picker under Microphone, "Input 1" to "Input 18" with the driver's channel |
| 250 | names where it gives them, remembered per device UID. LiveKit's engine module |
| 251 | wires the microphone in as mono, and the conversion from a multi-channel |
| 252 | interface keeps its first input only, so a microphone on input 2 used to send |
| 253 | silence. The module's observer now sets the input unit's channel map |
| 254 | (`kAudioOutputUnitProperty_ChannelMap`) so every channel it hands the engine |
| 255 | is the chosen input, and connects nothing, leaving the module's own mono |
| 256 | connection in place (`InputChannel.swift`). Changing it mid-call takes the |
| 257 | module's input away and gives it back (`setEngineAvailability`), which |
| 258 | rewires the microphone with the new map — a moment's gap, as a device switch |
| 259 | has. Since 0.3.8 the map is strictly opt-in (`InputChannelMap.channel`): it is |
| 260 | set only for an input other than the first, chosen on a microphone chosen by |
| 261 | name that has more than one input. On the system default, or any ordinary |
| 262 | microphone, the unit is left exactly as the module wired it — 0.3.3's |
| 263 | capture — and the picker is not shown. |
| 264 | - **A chosen microphone changing rate** (a Bluetooth headset moves to its |
| 265 | hands-free 16 or 24 kHz the moment its microphone opens) is listened to, and |
| 266 | a running engine wired at another rate is rebuilt (`RateRule`, at most once |
| 267 | every three seconds). On the system default the module follows the device |
| 268 | itself, as it always has. |
| 269 | |
| 270 | **How the engine is put on a device** (`AudioDevices.swift`, 0.3.7). Up to |
| 271 | 0.3.6 the app chose through LiveKit's module (`inputDevice = …`). Paul's |
| 272 | Scarlett 18i8 was not picked up at launch and would not stay chosen, and the |
| 273 | module's source (webrtc-sdk `m150_release`, `audio_engine_device.mm`) and a |
| 274 | probe of it show four reasons, any one of them enough: |
| 275 | |
| 276 | 1. The module lists only "default" until the first peer connection |
| 277 | initialises it, and raises no device-update callback when it does. At |
| 278 | launch the remembered Scarlett was missing from a one-entry list, so the |
| 279 | call fell back with "Scarlett 18i8 USB isn't connected; using …", and the |
| 280 | speaker's note named the Scarlett as the fallback because it was the |
| 281 | default output. |
| 282 | 2. Its list leaves out every virtual or aggregate device, and any device whose |
| 283 | input streams have an unknown terminal type while it also has outputs. |
| 284 | 3. It wires the microphone in with the *previous* device's format and only |
| 285 | then moves the engine's unit to the chosen device, so a device at another |
| 286 | sample rate starts silent or not at all. A start that fails has already |
| 287 | released the old engine, so the call is left with no audio. |
| 288 | 4. `trySetInputDevice`/`trySetOutputDevice` answer YES on failure and NO on |
| 289 | success, and `inputDevice`/`outputDevice` always read back nil. |
| 290 | |
| 291 | So the module is left on its default and never asked to choose. The pickers |
| 292 | list CoreAudio's devices (ready at launch, virtual and aggregate devices |
| 293 | included), and the module's "will enable" hook puts the engine's unit on the |
| 294 | chosen device before the module reads a format from it. On a Mac, |
| 295 | AVAudioEngine's input and output are one audio unit (measured: by default it |
| 296 | runs on macOS's private `CADefaultDeviceAggregate` of the two defaults), so a |
| 297 | microphone and speaker that are different devices are joined in a private |
| 298 | aggregate of the app's own: microphone first (AVAudioEngine will not start an |
| 299 | aggregate whose first device has no input), the speaker's clock leading, the |
| 300 | microphone drift-corrected. A microphone with outputs of its own has the |
| 301 | engine's output mapped past them. With nothing chosen the engine is left |
| 302 | exactly as before, on macOS's own default. A change mid-call rebuilds the |
| 303 | engine (both sides taken away and given back), a moment's gap in both |
| 304 | directions. The device the unit is actually on is read back as the engine |
| 305 | starts, and is what the readout names. |
| 306 | |
| 307 | **Test audio** (home screen, and the device panel in a call) meters the |
| 308 | capture hook — before noise reduction and after it — and records from the |
| 309 | hook's sink. Before a call it builds a CallAudio as a call does and sends its |
| 310 | microphone track between two in-process peer connections that are never |
| 311 | connected, then asks the engine module to record (`initRecording`, |
| 312 | `startRecording`: what WebRTC's first send stream calls). That last step is |
| 313 | 0.3.8's. WebRTC starts a send stream, and with it the microphone, only on a |
| 314 | connection whose transport has been writable (`BaseChannel::IsReadyToSendMedia_w`, |
| 315 | `was_ever_writable_`), so in 0.3.6 and 0.3.7 the test never opened the |
| 316 | microphone on any Mac: an empty meter and "We can't hear anything from …" for |
| 317 | every device (Paul's AirPods, 2026-09-26). The interop harness passed because |
| 318 | in manual rendering the module runs its input whenever its output runs; it now |
| 319 | also runs the start on the real device module, with 0.3.7's start as a control |
| 320 | that must never ask for the microphone. Calls themselves were not affected: |
| 321 | their connections are connected. |
| 322 | |
| 323 | **Nothing in the audio start runs on the main thread** (0.3.9). Paul's |
| 324 | 0.3.8, on a MacBook Air with AirPods Pro: "after I open the Test window, |
| 325 | there's a pause, then a beachball, then it works." A main-thread watchdog put |
| 326 | numbers on it: building the factory and its engine module (100–135 ms) and |
| 327 | logging the module's first device list (61–68 ms); applying the loopback's |
| 328 | answer, which waits for the engine's playout to start (188–229 ms); and |
| 329 | `initRecording`, which waits while the engine starts with the microphone — for |
| 330 | a Bluetooth headset, while it moves from A2DP to its hands-free profile, one to |
| 331 | two seconds. Every call into WebRTC's objects is a blocking hop to its own |
| 332 | threads, and while the worker thread is starting the engine every one of them |
| 333 | waits: the module's `isRecording`, `isPlaying` and device lists, the factory's |
| 334 | processing state, a track's `isEnabled` (mute) and its processing options, |
| 335 | 0.9–1.2 s each with the switch simulated. So all of it runs on one serial |
| 336 | queue (`AudioWork`), in the order asked; the main thread reads the module's |
| 337 | state from `ModuleState`, which waits at most 5 ms and otherwise gives the last |
| 338 | answer. The window opens at once and says **Starting microphone…** beside a |
| 339 | spinner — **Switching Paul's AirPods Pro to microphone mode…** for a Bluetooth |
| 340 | headset not yet at its hands-free rate, because that wait is physics, not a |
| 341 | fault — and the meter comes alive when the microphone has started. Closing it |
| 342 | partway cancels cleanly: the stop is queued behind the start, so what the |
| 343 | start opens is closed (`MicStart`). A call's join builds its media the same |
| 344 | way (`MediaEngine.make`), and its connections' descriptions, candidates, |
| 345 | statistics, sender parameters and closes go through the same queue |
| 346 | (`PeerCalls`). Measured by `CallsInterop --audio-start-stall` on an M1 Max, |
| 347 | longest main-thread stall, before → after: Test audio 142–716 ms → 0–5 ms; |
| 348 | with a 1.5 s Bluetooth switch simulated 1,536–1,657 ms → 1–7 ms; closed |
| 349 | mid-start → 0–8 ms; a call's audio start with the switch 1,322–1,514 ms → |
| 350 | 21–43 ms. |
| 351 | |
| 352 | The verdict names the device the microphone really is, with its connection |
| 353 | and rate ("Paul's AirPods Pro (Bluetooth, 24 kHz)"), and tells apart "no sound |
| 354 | is arriving" (no buffers at all: the microphone was asked for and nothing came |
| 355 | back) from "sound is arriving, but it is silent". Bluetooth rates, which the |
| 356 | suppressor passes through, are metered and recorded like any other. |
| 357 | |
| 358 | **Copy audio diagnostics**, in Test audio (in the panel's ⓘ it is **Copy call |
| 359 | diagnostics**, with the screen's account added), puts text on |
| 360 | the clipboard to paste into a message: the devices chosen and what they |
| 361 | resolve to, every CoreAudio device with its rate and transport, the engine |
| 362 | module's state and last few engine events with their formats, the capture |
| 363 | hook's rate and buffer counts, the processing report, the last ten seconds of |
| 364 | meter readings in dBFS, and the last 50 lines of the audio log (which is also |
| 365 | kept in memory, the last 200 lines). Never any audio. |
| 366 | |
| 367 | Calls 0.2.1 (build 3) routed audio differently: its own CoreAudio audio |
| 368 | device (`CallsAudio`, one pair of HAL units per call) behind WebRTC's |
| 369 | custom-audio-device constructor. That constructor takes no audio processing |
| 370 | module, so it cannot carry the noise suppressor; the rules above are that |
| 371 | build's, carried onto LiveKit's module, and `CallsAudio` was not kept. |
| 372 | |
| 373 | The camera list includes external and Continuity cameras (the Mac app |
| 374 | declares `NSCameraUseContinuityCameraDeviceType`) and refreshes every second |
| 375 | while the device panel is open. A camera switch that is refused tries to |
| 376 | restore the previous camera, and the picker returns to the camera actually in |
| 377 | use. |
| 378 | |
| 379 | Apple's voice processing is **not** allowed on Mac |
| 380 | (`setPlatformVoiceProcessingAllowed(false)`). Left allowed, the engine module |
| 381 | resolves echo cancellation, noise suppression and gain to Apple's coupled |
| 382 | voice-processing unit; with it off, WebRTC's own software processing runs, as |
| 383 | it always has on Mac. One consequence: macOS offers its **Mic Mode** menu |
| 384 | (Voice Isolation and the rest) only to apps on voice processing, so it is not |
| 385 | offered to Calls on Mac. The iPhone still runs on the voice-processing unit. |
| 386 | |
| 387 | The native/browser gate cannot open a microphone (a test process has no |
| 388 | permission to), so `CallsInterop` gives the engine module a synthetic voice |
| 389 | through manual rendering mode, as it gives the capturer synthetic frames; no |
| 390 | device is opened on either side of it. |
| 391 | |
| 392 | **Reduce background noise** (in the device panel, on by default, remembered) |
| 393 | replaces WebRTC's classic noise suppressor with a neural one, for the noises |
| 394 | people actually hear: typing, wind, a room's echo. WebRTC's capture |
| 395 | processing runs high-pass filter, echo canceller, noise suppressor and gain |
| 396 | control, then hands each 10 ms frame to a capture post-processing hook |
| 397 | (`RTCDefaultAudioProcessingModule`); the suppressor runs there, after the |
| 398 | echo canceller has removed the far side. `ProcessingPlan` holds the rules: |
| 399 | |
| 400 | - WebRTC's noise suppressor is off while ours runs, and ours never runs |
| 401 | beside Apple's Voice Isolation (it stands aside when that is the active |
| 402 | mic mode). |
| 403 | - WebRTC's gain control runs *before* the hook, so it would lift the room |
| 404 | with a quiet voice and hand the suppressor more noise to remove. It is off |
| 405 | while ours runs; `SpeechGain` lifts the voice after the noise is gone |
| 406 | instead — only upward, at most 12 dB, learning only from frames the model |
| 407 | calls speech, about 1 dB a second, with a peak limit. |
| 408 | - The models run at 48 kHz only. At any other capture rate (a Bluetooth |
| 409 | headset's 16 kHz) frames pass through untouched and WebRTC's own |
| 410 | suppressor and gain come back on the Mac. |
| 411 | - Off means no suppression at all; WebRTC's gain control comes back. |
| 412 | - On iPhone Apple's echo canceller and its noise suppressor are one switch |
| 413 | in the voice-processing unit, and the phone has always run both; that |
| 414 | pairing stays. Apple's gain control is separate and comes off while ours |
| 415 | runs. |
| 416 | |
| 417 | The audio readout says which suppressor is running and at what share of a |
| 418 | core, and what WebRTC reports as running for echo cancellation, noise |
| 419 | suppression and gain — what it reports, not what was asked. WebRTC skips the |
| 420 | hook while the microphone is muted, and the readout says so. WebRTC opens the |
| 421 | microphone only once a connection is sending, so a person alone in a call has |
| 422 | none running yet; until then the processing module reports its construction |
| 423 | default (16 kHz, everything off), and on Mac the readout says "The microphone |
| 424 | has not started yet" instead of repeating that. Every initialisation of the |
| 425 | capture hook is reported to the plan, a repeat of the same rate included, so a |
| 426 | microphone really capturing at 16 kHz brings WebRTC's own suppressor back. |
| 427 | |
| 428 | **On Mac the suppressor is DeepFilterNet 3**; on iPhone it is RNNoise. |
| 429 | |
| 430 | - **DeepFilterNet 3** (Rikorose/DeepFilterNet's libDF, MIT or Apache-2.0; |
| 431 | notice in `LICENSE.deepfilternet`, bundled in the Mac app) is markedly |
| 432 | better on non-stationary noise — typing, wind, voices in the next room. It |
| 433 | takes 480-sample hops at 48 kHz in ±1 floats, so samples are scaled by |
| 434 | 1/32768 on the way in and back on the way out, and adds about 30 ms of |
| 435 | latency. `scripts/build-deepfilter.sh` builds `native/build/DeepFilter.xcframework` |
| 436 | (static; `aarch64-apple-darwin`, `aarch64-apple-ios`, `aarch64-apple-ios-sim`) |
| 437 | with a pinned rustup toolchain, cbindgen for the header, and the C API |
| 438 | (`--features capi`), and copies the standard model |
| 439 | (`DeepFilterNet3_onnx.tar.gz`, 8 MB) beside it; the Mac bundle carries the |
| 440 | model. Needs rustup (https://rustup.rs). The library is linked on Mac only. |
| 441 | The model takes about 0.3 s to load (on an M1 Max; longer on an Air), so |
| 442 | one is loaded three seconds after launch, at utility priority, and the |
| 443 | first Test audio or call takes it ready-made (`DeepFilterPreload`). A |
| 444 | loaded model is about 88 MB resident, so only one is kept spare: taking it |
| 445 | does not load another at once (a call would then hold two), a new spare is |
| 446 | loaded when its user is done, and a spare nobody has taken for fifteen |
| 447 | minutes is let go. Without a spare a call starts on RNNoise and moves to |
| 448 | DeepFilterNet when one has loaded, off the main thread, as before; a model |
| 449 | is never handed from one user to the next, because its state holds the |
| 450 | last audio it heard and libDF has no way to clear it. A missing model |
| 451 | leaves it on RNNoise. (RNNoise itself is made in under a millisecond and is |
| 452 | not preloaded.) `CALLS_DEEPFILTER_MODEL` points at a model file |
| 453 | outside a bundle (the tests and the interop harness). |
| 454 | - **RNNoise** (xiph, BSD-3; notice in `LICENSE.rnnoise`, bundled in both apps): |
| 455 | 480-sample frames at 48 kHz in WebRTC's own ±32768 scale. Its model weights |
| 456 | arrive as a generated C file of 78 MB, so it is not a source target: |
| 457 | `scripts/build-rnnoise.sh` fetches the pinned commit and the checksummed |
| 458 | model and builds `native/build/RNNoise.xcframework` (static, Mac, iPhone |
| 459 | and simulator). |
| 460 | - Both are limited to 30 dB of attenuation: unlimited, a room goes to digital |
| 461 | silence between words, which is heard as the line dropping. DeepFilterNet |
| 462 | has the limit built in; RNNoise gets it by mixing the untouched input back |
| 463 | in at -30 dB. |
| 464 | - Measured on an M1 Max (fixture test, isolated): RNNoise 0.69–0.76 ms and |
| 465 | DeepFilterNet 3 1.16–1.23 ms per 10 ms frame, i.e. about 7% and 12% of one |
| 466 | core. In the native/browser gate, as wall-clock share of the audio thread |
| 467 | with Chromium busy on the same machine: RNNoise median 12.9%, DeepFilterNet |
| 468 | median 15.1% (13–27%). DeepFilterNet has not been measured on an iPhone, |
| 469 | which is why the phone stays on RNNoise; the xcframework already carries the |
| 470 | iOS slices, so trying it is a Package.swift condition and a bundled model. |
| 471 | |
| 472 | None of these libraries is committed: `build-mac.sh` and `build-iphone.sh` |
| 473 | run all three scripts (bridge, RNNoise, DeepFilter), each cached against its |
| 474 | pins; run them yourself before `swift test`. |
| 475 | |
| 476 | Choosing a Bluetooth microphone on iPhone puts the call on the hands-free |
| 477 | profile, which is narrowband in both directions. That used to happen |
| 478 | silently to every call the moment any headset connected, because |
| 479 | `.allowBluetoothHFP` was set unconditionally; it is now set only while a |
| 480 | Bluetooth input is the chosen one, and the panel says what it costs. |
| 481 | |
| 482 | On iPhone and iPad the panel also names Apple's **Mic Mode** — Standard, |
| 483 | Voice Isolation or Wide Spectrum — and **Change Mic Mode** opens the system |
| 484 | picker. Voice Isolation is the best noise suppression the device has (wind, |
| 485 | typing, room echo), and it reaches the call because WebRTC's iOS audio device |
| 486 | runs on Apple's voice-processing I/O unit. Apps cannot set the mode, only show |
| 487 | it and open the picker; when the route cannot honour the chosen mode, the panel |
| 488 | names the one actually in use. The Mac app does not offer it: WebRTC's macOS |
| 489 | audio device does not use Apple's voice processing, so the mode would not reach |
| 490 | the call. |
| 491 | |
| 492 | The **audio readout** separates the two ways received audio goes wrong, |
| 493 | which is the distinction a person reporting "it growled" cannot make: |
| 494 | concealed samples are loss and sound like dropouts, while inserted and |
| 495 | removed samples are the jitter buffer rescaling the signal to track the |
| 496 | sender's clock — and a stretched voice is what a growl is. The device pickers let you compare routes while watching those counters. |
| 497 | |
| 498 | **Shared screens** from anyone in the call — a browser or the other app — |
| 499 | take the stage as a tile of their own, captioned "Name’s screen" exactly as |
| 500 | the browser captions it, while the sharer's camera moves to a thumbnail |
| 501 | beside yours. Nothing on the wire says which m-line is a screen; the app |
| 502 | works it out with the browser's own rules (spec §5.4, `classifyTrack` in |
| 503 | `room.js`, ported in `ScreenShare.swift` with its tests), so the two never |
| 504 | disagree about whether they are looking at a face or a document. The tile |
| 505 | goes on whichever of §5.4's two endings arrives first: the `share {on:false}` |
| 506 | message, or the stopped transceiver taking its receiver away. Each was |
| 507 | checked alone against a browser when this was written (the other one |
| 508 | disabled); the native gate checks them together. |
| 509 | |
| 510 | **Share your screen** on the Mac opens the system's own picker |
| 511 | (`SCContentSharingPicker`, macOS 14) for a display or a window. What a person |
| 512 | picks there is what they have agreed to share, so Calls asks for no Screen |
| 513 | Recording permission, sends nobody to System Settings, and needs no Info.plist |
| 514 | key — macOS has no usage string for screen capture — and the same path works in |
| 515 | the App Sandbox. Calls' own windows are left out of the picker, and from a shared |
| 516 | display too on macOS 14.4 and later, where the display is rebuilt excluding the |
| 517 | app so that windows opening later (the floating mini window among them) stay |
| 518 | out as well. The screen goes to every connection on its own `sendonly` |
| 519 | transceiver in its own stream, exactly as the browser sends one, announced with |
| 520 | `share {on:true}`; stopping stops the transceiver and renegotiates, so the far |
| 521 | side's track ends rather than freezing, and says `share {on:false}`. The menu |
| 522 | bar's stop control and a shared window closing end it the same way as the |
| 523 | button, and so does **Stop sharing** in the "You are sharing your screen" |
| 524 | notice itself (0.4.3; the iPhone's and the browser's notices have it too), which |
| 525 | stays on screen while the controls fade. Frames are capped at 1920 on the long side and 30 a second. |
| 526 | |
| 527 | **Optimise shared screen for** (in the device panel, remembered, applied live |
| 528 | to a share in progress) decides what gives way when the uplink or the encoder |
| 529 | cannot carry everything: **Balanced** (the default) holds the frame rate and |
| 530 | sends full size while the link allows; **Text and detail** holds resolution |
| 531 | and lets frames fall; **Motion and video** holds the frame rate at 1280 on the |
| 532 | long side at most. They are sender parameters (`ScreenSendPolicy`), so nothing |
| 533 | renegotiates. While a screen goes out it asks for H.264 first (Apple's |
| 534 | hardware encoder), with every other codec still offered, and has high priority; |
| 535 | the camera beside it is capped at 500 kbit/s and 360 lines, and given its |
| 536 | defaults back when the share stops. While sharing, the panel shows one line on |
| 537 | how the screen is actually leaving, e.g. "Screen: 1920×1080 at 8 fps, limited |
| 538 | by bandwidth, 1.2 Mbit/s, direct (H264 hardware)", and a second line with the |
| 539 | encoded and sent frame rates, keyframes made and asked for, and what the screen |
| 540 | watchdog is doing. The Mac capture keeps six surfaces in its queue (three, in |
| 541 | 0.4.x, left too few once the encoder and the repeat were holding theirs, and a |
| 542 | capture with every surface held stops dead), drops a frame that arrives more than |
| 543 | 100 ms late, and repeats the last frame once a second while the screen is still, |
| 544 | so a keyframe can be made for a receiver that asks for one. A screen whose source |
| 545 | is not a screencast (Motion, or a share started in it) is kept within H.264 level |
| 546 | 3.1's 1280×720 worth of blocks: Chromium answers H.264 at that level, and the |
| 547 | Mac's encoder refused larger frames outright (spec §15.21). The browser offers the |
| 548 | same three choices, as `contentHint`, `degradationPreference` and sender |
| 549 | parameters. |
| 550 | |
| 551 | **A screen that stops is put right without anyone sharing again** (spec §15.21, |
| 552 | after Paul's "my screen froze on her end … I had to stop and re-share", 28 Sep). |
| 553 | The sharer's watchdog (`ScreenWatchdog`) looks once a second: a capture that |
| 554 | reports nothing for three seconds is restarted on the same content; a connection |
| 555 | that takes frames in and encodes or sends none for five seconds (past its first |
| 556 | ten, and not held back by the link) gets a keyframe, then a fresh encoder (the screen a third |
| 557 | smaller for thirty seconds), then a new transceiver, three seconds apart. A receiver — app or browser — whose view of someone's screen has |
| 558 | decoded nothing for four seconds says so in its `media` report |
| 559 | (`screenStalled`), the sharer sends that connection keyframes (and the fuller |
| 560 | steps only if its own screen has stopped leaving too), and shows |
| 561 | "Vicky's view of your screen stalled — recovering" while it does. Each step is in |
| 562 | `log show --predicate 'subsystem == "net.amadan.oficina.calls" && category == |
| 563 | "screen"'` and in **Copy call diagnostics** (the panel's ⓘ; Test audio's is still |
| 564 | **Copy audio diagnostics**). The apps restart ICE only when a relay credential is |
| 565 | due for renewal, not on every stream resume as 0.4.x did. |
| 566 | |
| 567 | **When keyframes are not enough** (spec §15.22, after Paul's "still seeing issues |
| 568 | with screen sharing after a while", 30 Sep, the notice sitting at "recovering"): |
| 569 | a receiver still reporting six seconds after the first keyframe gets its screen |
| 570 | rebuilt — a new transceiver for that person alone, so a new encoder here and a new |
| 571 | receiver and decoder there, at the same size and mode (a report never shrinks a |
| 572 | Text and detail share) — and the notice says "Restarting your share for Samuel…". |
| 573 | Still reported ten seconds later, everyone's screen is shared again from the same |
| 574 | capture, with no picker: what Stop and Share did by hand. Ten seconds after that |
| 575 | the notice says "Couldn't restore Samuel's view — try Stop and Share again". A |
| 576 | rebuild at most once a minute per person, a share again at most once every five |
| 577 | minutes, and neither unless frames actually left for that person since the last |
| 578 | step. The receiver hands a stalled screen's picture to its view again four seconds |
| 579 | in, logs what its statistics say is still arriving (packets, frames received, |
| 580 | decoded, dropped, keyframes, PLI, the decoder), holds its report for up to three |
| 581 | minutes, and counts any change in the decoded count as movement (a rebuilt screen's |
| 582 | count starts again from zero, which 0.4.3 read as still stalled). The iPhone's |
| 583 | broadcast repeats its last frame once a second while the screen is still, as the |
| 584 | Mac's capture does, so a still iPhone screen is no longer reported as stalled. The |
| 585 | browser does all of this too (its sharer's rebuild and share again included), and |
| 586 | has **Copy call diagnostics** at the foot of its Devices panel. |
| 587 | |
| 588 | "Reconnecting…" is said only for a problem that lasts (spec §15.23): the |
| 589 | signalling stream down 4 s, a connection `disconnected`/`failed` 3 s, or no |
| 590 | audio for 2 s from someone unmuted — that one on their tile, "Reconnecting to |
| 591 | Samuel…" — and it stays a second past the recovery. The stream's own renewal |
| 592 | every few minutes is never said. `ReconnectWatch`; the `connection` log |
| 593 | category. |
| 594 | |
| 595 | The gate for it wedges a receiver's screen decoder, which keyframes cannot mend: |
| 596 | `screen-decoder-wedge` in `CallsInterop` (run with `CALLS_DECODER_WEDGE=1`; a |
| 597 | wrapping decoder factory that only the gate turns on), and in Chromium an encoded |
| 598 | transform that swallows every frame. `wedge`, `wedge-sticky` and `reshare` in |
| 599 | `make native-pair`, and `wedge`, `wedge-sticky`, `web-wedge` and `web-wedge-sticky` |
| 600 | in `make native-browser`, hold the ladder to restoring the view within 30 s, or to |
| 601 | saying it cannot. `soak` and `web-soak` (`SCREEN_SOAK_SECONDS`) and the pair's |
| 602 | `soak` (`SCREEN_PAIR_SOAK`) are the long shares, run by hand. |
| 603 | |
| 604 | The numbers behind the modes come from `TestNativeMacScreenMotion` (in |
| 605 | `make native-browser`): the app shares a 1080p page scrolling at 30 fps with a |
| 606 | window dragged across it, and Chromium measures what arrives, with the app's |
| 607 | uplink estimate open and capped at 1.5 Mbit/s. Before this, a 1.5 Mbit/s uplink |
| 608 | delivered 2 fps at full size. Balanced and motion must now arrive at 15 fps or |
| 609 | more, and detail must keep 1920×1080. |
| 610 | |
| 611 | On **iPhone** the whole screen is shared through a ReplayKit broadcast |
| 612 | extension, which hands its frames to the app over a socket in an App Group |
| 613 | container. See [ios/README.md](ios/README.md#sharing-the-screen) for how that |
| 614 | works and what signing it needs. |
| 615 | |
| 616 | **Knocks** are announced when someone asks to be let in while the app is |
| 617 | not frontmost. The admission queue is already on screen for a member who is |
| 618 | looking at the call; this is for the member who is in another Space, |
| 619 | another app, or has the phone in a pocket. |
| 620 | |
| 621 | A knock also reaches a member whose app is **closed**, where the server |
| 622 | has an Apple push key. The app registers its device token at join |
| 623 | (`POST /native/devices`); the hub calls back when someone begins waiting; |
| 624 | the server pushes to every member who was *not* admitted in that room at |
| 625 | that moment, because the ones who were can already see the queue. |
| 626 | |
| 627 | Without a key the server sends nothing and says nothing, which is where |
| 628 | this app was before push existed: knocks still reach everyone in the room |
| 629 | over their own streams. It is configured with `CALLS_APNS_KEY` (the .p8 |
| 630 | as Apple issues it, via `carlos secrets set`), `CALLS_APNS_KEY_ID`, |
| 631 | `CALLS_APNS_TEAM_ID`, `CALLS_APNS_TOPIC_IOS`, `CALLS_APNS_TOPIC_MACOS` |
| 632 | and optionally `CALLS_APNS_SANDBOX`. All five of the first are required |
| 633 | together — a half-configured push is refused at boot rather than |
| 634 | discovered on the first knock. |
| 635 | |
| 636 | Someone who arrived while the queue was visible is never announced later, |
| 637 | so tabbing away does not produce a pile of notifications about people who |
| 638 | have been waiting a while. Only an arrival during an absence is news, and |
| 639 | leaving the queue and knocking again is heard again. |
| 640 | |
| 641 | On Mac, **switching to another app during a call** puts the call in a small |
| 642 | floating window, bottom right by default, and coming back to Calls restores the |
| 643 | call window as it was. It is the first setting in **Calls → Settings…**, on by |
| 644 | default, with the corner beside it; dragging the window snaps it to the nearest |
| 645 | corner and remembers that one. **Mini window placement** can instead be |
| 646 | **Anywhere I put it**: the window stays exactly where it is dropped, remembered |
| 647 | per display (by display ID, as an offset from that display's visible frame, so |
| 648 | rearranging displays does not lose it) and clamped back on screen when a display |
| 649 | changes. Holding **⌥ Option** while dragging places it freely even when snapping |
| 650 | to corners, for the rest of that call only. It resizes from its edges and |
| 651 | corners at 16:9, from 200 pt wide up to 40% of the screen's width, and the size |
| 652 | is remembered; snapping keeps it (`MiniWindowResize`, done by the window itself |
| 653 | rather than AppKit's `.resizable`). Clicking the window brings Calls back; |
| 654 | hovering shows Collapse, Mute and Leave. **Collapse** shrinks it to a tab |
| 655 | against the nearest screen edge showing your microphone state and who the call |
| 656 | is with (the other person's name, or a count); clicking the tab or dragging it |
| 657 | away from the edge opens the mini window again, and double-clicking it brings |
| 658 | the call window forward. Collapsed lasts for the rest of the call; every call |
| 659 | starts expanded. The tab is the same window, so it floats, follows Spaces and |
| 660 | stays out of screen captures exactly as the mini window does. There is no |
| 661 | speaking light: nothing in the app measures audio level live. |
| 662 | |
| 663 | **Space** toggles the microphone while the call window (or the mini window) is |
| 664 | the key window, unless a text field or a control has the keyboard, where Space |
| 665 | is typing or the control's own press; a held Space toggles once. **Call → Mute |
| 666 | Microphone** (⌘⇧M) is the same toggle. The browser room does the same (never in |
| 667 | an input, textarea, select, contenteditable, button, link or other control, and |
| 668 | announced through the polite live region), and on iPhone and iPad a hardware |
| 669 | keyboard's Space does it through a key command that yields to text editing and |
| 670 | focus. Push-to-talk (holding Space while muted) is not built. |
| 671 | |
| 672 | **Call → Copy Invite Link** (⌘⇧I), beside Mute, puts the meeting's invite link |
| 673 | on the clipboard and says "Invite link copied" briefly over the stage (and to |
| 674 | VoiceOver), from the call window or the mini window, with the controls shown or |
| 675 | not. It asks the server the way the Share button does, which returns the |
| 676 | meeting's current sealed invitation and never replaces or revokes one. It is |
| 677 | off outside a call; a guest pressing it is told only members can share. An |
| 678 | iPad's hardware keyboard has the same shortcut. The browser room does not: |
| 679 | ⌘⇧I is Chrome's developer tools. It shows what Speaker view's stage shows (0.4.0), whichever layout the call |
| 680 | window is in: whatever you pinned, otherwise the newest shared screen, otherwise |
| 681 | whoever is speaking (see [DESIGN.md](DESIGN.md#speaker-view-and-your-own-order--2026-09-27-040)), |
| 682 | otherwise the first other participant in your order, with their muted and |
| 683 | camera-off state, or your own camera while you are alone. It is not |
| 684 | shown for a minimised call window, and ⌘H hides it with the rest of the app. The |
| 685 | call window is hidden by transparency rather than ordered out, because SwiftUI |
| 686 | treats an ordered-out window as closed, which would leave the call. The mini |
| 687 | window is excluded from screen capture: a shared display leaves out the whole |
| 688 | Calls application (macOS 14.4 and later), and the window also sets |
| 689 | `sharingType = .none`, so it never appears in a screen this app shares. |
| 690 | |
| 691 | Members can use **Share** in the call to retrieve the current invitation, copy |
| 692 | it, or open the system sharing menu. This needs the server's |
| 693 | `POST /m/{sqid}/share` endpoint and the encrypted-invitation migration. |
| 694 | |
| 695 | There are no embedded webviews. SwiftUI renders the app and native WebRTC handles |
| 696 | media. The shared Go core waits for a five-minute, one-use authorization bound |
| 697 | to a proof secret held only by the app. Browser approval creates a separate, |
| 698 | ordinary framework session, preserving the existing identity method and Home |
| 699 | assertion authentication time. No browser cookie is copied, and no new bearer |
| 700 | authentication type is introduced. A member's session is kept in the Keychain, |
| 701 | as described above. |
| 702 | |
| 703 | **The server must include this branch's native authorization endpoints.** An |
| 704 | older server cannot complete native sign-in; the app reports that limitation. |
| 705 | The production handoff endpoints were deployed in `86ed9f3`. In-progress handoffs expire on |
| 706 | server restart. There are at most 256 pending handoffs per server process. |
| 707 | |
| 708 | `make native-mac` builds for the host Mac and signs the local `.app` ad hoc. |
| 709 | Developer ID signing, notarization and App Store packaging remain follow-up work. |
| 710 | The iPhone target, device builds and Safari tests are described in |
| 711 | [ios/README.md](ios/README.md). |
| 712 | |
| 713 | ## Quality reports and Report a problem |
| 714 | |
| 715 | The apps send a connection-quality report every 30 seconds in a call and one |
| 716 | at leave, and "Report a problem" (the ⋯ menu, the ⓘ panel, and the screen a |
| 717 | call ends on) sends the call diagnostics with a note. Spec §15.26 says exactly |
| 718 | what is in them and what never is; Settings has the disclosure and the |
| 719 | "Send anonymous quality reports" switch. They land in the server's log as |
| 720 | `calls: quality` and `calls: problem report` JSON lines under random call and |
| 721 | peer tags, kept for the platform's seven days. To read them: |
| 722 | |
| 723 | ```sh |
| 724 | # The call at 15:40 today (Europe/Dublin), summarised |
| 725 | go run ./cmd/quality-summary --since 24h --at 15:40 |
| 726 | # A reference someone was shown, with its diagnostics |
| 727 | go run ./cmd/quality-summary --problem K7Q2M9PD --since 7d |
| 728 | # The raw lines of one call |
| 729 | carlos logs --app calls --account oficina --since 24h -n 5000 --field call=<tag> |
| 730 | ``` |
| 731 | |
| 732 | [docs/operations/quality.md](../docs/operations/quality.md) has the line |
| 733 | shapes, more queries, and the kill switch (`CALLS_QUALITY_REPORTS=off`). |
| 734 | |
| 735 | ## Shared code |
| 736 | |
| 737 | - `protocol/`: the server and client use the same Go wire types. The hub keeps |
| 738 | aliases so authorization and admission behavior remain in the server. |
| 739 | - `callscore/`: browser sign-in handoff, bounded SSE parsing, room state, per-target ordered sends, |
| 740 | generation fences, retry, resume and ICE credential fetching. No UI, database, |
| 741 | or media dependency. |
| 742 | - `native/bridge`: a narrow gomobile Objective-C interface consumed by Swift. |
| 743 | - `native/apple/Sources/CallsKit`: Swift adapter and native WebRTC media. The |
| 744 | platform capture/UI portions are separate from the Go protocol implementation. |
| 745 | - `native/apple/Sources/CallsMac`: SwiftUI app and system-browser authentication host. |
| 746 | - `native/apple/Sources/CallsInterop`: a separate synthetic-media test executable; |
| 747 | it is not bundled with Calls.app. |
| 748 | |
| 749 | Go Mobile's cgo requirement is isolated in `native/go.mod` and the native build |
| 750 | script. The existing server continues to build with `CGO_ENABLED=0`. WebRTC is |
| 751 | LiveKit's build of Google WebRTC, `livekit/webrtc-xcframework` pinned exactly to |
| 752 | 150.7871.02 (M150); SwiftPM verifies its published artifact checksum. It replaced |
| 753 | stasel/WebRTC 152 because it carries what that build does not: an audio device |
| 754 | module on macOS (device lists and selection) and a capture post-processing hook |
| 755 | for noise suppression. Its Objective-C symbols are prefixed `LK`; CallsKit's |
| 756 | `WebRTCNames.swift` aliases the old `RTC…` names once, so code and open branches |
| 757 | keep using them and change only their import (`import LiveKitWebRTC`). |
| 758 | Go Mobile is also pinned; so are RNNoise and DeepFilterNet (see above). The bridge includes Mac, iPhone and iOS Simulator |
| 759 | slices. Generated frameworks, Xcode projects and binaries are ignored. |
| 760 | |
| 761 | ## Verification |
| 762 | |
| 763 | ```sh |
| 764 | CGO_ENABLED=0 make ci |
| 765 | make race |
| 766 | CGO_ENABLED=0 make browser |
| 767 | make native-test |
| 768 | make native-browser |
| 769 | make native-pair |
| 770 | CGO_ENABLED=0 go test -tags browser -run TestNativeBrowserAuthentication ./internal/callstest |
| 771 | ``` |
| 772 | |
| 773 | `TestNativeMacAudioStartStall` (in `make native-browser`) runs |
| 774 | `CallsInterop --audio-start-stall`: a watchdog posts to the main queue every |
| 775 | 5 ms and times the answer while Test audio starts (plainly, with a 1.5 s |
| 776 | Bluetooth switch simulated by holding the engine's start on WebRTC's worker |
| 777 | thread, and closed partway through) and while a call's audio starts with the |
| 778 | same switch, all on the real engine module. Each must hold the main thread |
| 779 | under 100 ms; a control that blocks it for 300 ms on purpose must be seen. |
| 780 | |
| 781 | `swift test` needs `native/scripts/build-bridge.sh`, |
| 782 | `native/scripts/build-rnnoise.sh` and `native/scripts/build-deepfilter.sh` to |
| 783 | have run. Its suppressor tests feed |
| 784 | `Tests/CallsKitTests/Fixtures/speech-48k.wav` (macOS's own speech |
| 785 | synthesiser, normalised to -23 dBFS, a second of silence ahead) mixed with |
| 786 | seeded hiss and key clicks through each real suppressor, and require the |
| 787 | noise-only lead-in to fall by at least 10 dB, the speech level to hold within |
| 788 | 6 dB, and 80% of voiced frames to survive. The frame adapter, the gain stage |
| 789 | and the processing plan have unit tests of their own. |
| 790 | |
| 791 | The native/browser gate requires the same Playwright/Chromium setup as |
| 792 | `make browser`, and working UDP. It creates two real magic-link sessions and a |
| 793 | meeting on an isolated server, starts Chromium and the separate native test |
| 794 | executable, then checks guest admission, actual incoming/outgoing media, clean departure and continuity through |
| 795 | ten-second SSE renewals, bidirectional chat and the native sender's accepted-message |
| 796 | history. Test speaker playback is disabled in both clients; audio checks use RTP |
| 797 | counters, not audible output. The native side's microphone is a synthetic |
| 798 | voice through the audio device module's manual rendering mode, so its capture |
| 799 | passes through WebRTC's real processing: the gate asserts the noise suppressor |
| 800 | processed frames, and that WebRTC reports its own suppressor and gain off and |
| 801 | its echo canceller on. `make native-browser` builds `CallsInterop` itself; |
| 802 | it used to run whichever binary an earlier build had left. It never needs production credentials or hardware |
| 803 | camera access. The Go Home test separately exercises a session minted through |
| 804 | Home assertions and explicit identity linking, including resume and meeting end. |
| 805 | |
| 806 | The native/browser gate also shares screens both ways: the browser shares (a |
| 807 | fake monitor, `--auto-accept-this-tab-capture`) and the app must caption and |
| 808 | decode it beside the camera, then drop it when it stops; the app shares a |
| 809 | synthetic screen and the browser must show it under the app's name, then see its |
| 810 | received track *end* — once by the button and once by the capture ending on its |
| 811 | own. `make native-pair` puts two apps in one call with no browser: a screen |
| 812 | shared before the other app joined, both endings, and both apps sharing at once. |
| 813 | `TestNativeMacScreenRecovery` (in `make native-browser`) and |
| 814 | `TestNativeMacScreenPairRecovery` (in `make native-pair`) share a screen |
| 815 | through relay renewals every twenty seconds and stream resumes every ten, a |
| 816 | capture that stops, an encoder that stops, and a receiver that reports a stall, |
| 817 | and require the watcher's screen to keep decoding or come back without a new |
| 818 | share; `SCREEN_PAIR_A`/`SCREEN_PAIR_B` put an older build on either side, and |
| 819 | `screen-display` shares this Mac's real display through ScreenCaptureKit and the |
| 820 | hardware encoder (Screen Recording permission needed). |
| 821 | It is separate because two unsigned apps on one Mac do not reliably connect at |
| 822 | all (see `TestNativeMacScreenShare`), which looks like the machine rather than |
| 823 | the feature. |
| 824 | |
| 825 | The native chat gate sends a sequence of browser messages and renders the shared |
| 826 | native chat view to `native/build/chat-incoming.png` for visual inspection of the |
| 827 | latest message. The window stays off-screen. Native chat follows new messages |
| 828 | and keeps its history visible during signaling reconnects. |
| 829 | |
| 830 | There is no claim that a simulator or synthetic test proves physical camera, |
| 831 | microphone permissions, audio-device switching, sleep/wake or Home passkeys. |
| 832 | Those need a real-device acceptance pass. So does screen sharing: the Mac's |
| 833 | system picker and the iPhone's broadcast extension cannot be driven by any gate |
| 834 | here. Background calling (other than while sharing the iPhone's screen) and |
| 835 | long-call TURN renewal parity are follow-up work. |
| 836 | |
| 837 | See [VERIFICATION.md](VERIFICATION.md) for the completed gate record and acceptance limits. |
| 838 | |
| 839 | ### App artwork |
| 840 | |
| 841 | The iPhone asset catalog and Mac application resource use the approved Oficina |
| 842 | Calls icon. Both are generated copies of Estilo's `brand/png/calls-1024.png`; |
| 843 | update them with `python3 brand/distribute.py calls /path/to/calls` from Estilo. |
| 844 | The Mac's icon comes from the iPhone's full-bleed square, the one master. |
| 845 | `native/scripts/bundle-mac.py` wraps it in an Icon Composer `AppIcon.icon` and |
| 846 | compiles it with `xcrun actool` into `Assets.car`, named by `CFBundleIconName`: |
| 847 | macOS 26 and later mask, light and tint it themselves, and actool flattens it |
| 848 | onto the 824-on-1024 grid for macOS 14 and 15. This needs Xcode 26 or later. |
| 849 | `native/scripts/generate-mac-icon.swift` derives `MacIcon.png`, the master |
| 850 | rounded onto that grid, for `CallsMac.icns` (`CFBundleIconFile`) and for the |
| 851 | Dock icon set at launch, which happens only before macOS 26 or outside a |
| 852 | bundle. Never feed it `Resources/AppIcon.png`, which is already on the grid: |
| 853 | padding it twice shrank the Dock icon to 664 of 1024 from 0.4.5. The Lucide |
| 854 | notice is retained in `native/LICENSE.lucide`. |
| 855 | |
| 856 | Third-party notices ship in the apps' Resources: |
| 857 | |
| 858 | | Notice | What | Mac | iPhone | |
| 859 | |---|---|---|---| |
| 860 | | `LICENSE.lucide` | Lucide icons | yes | yes | |
| 861 | | `LICENSE.webrtc` | WebRTC's own BSD licence, as LiveKit ships it in `LiveKitWebRTC.xcframework` | yes | yes | |
| 862 | | `PATENTS.webrtc` | WebRTC's additional IP rights grant (patents) | yes | yes | |
| 863 | | `LICENSE.rnnoise` | RNNoise | yes | yes | |
| 864 | | `LICENSE.deepfilternet` | DeepFilterNet's libDF | yes | no (not linked) | |
| 865 | | `LICENSE.deepfilternet-crates` | the 90 Rust crates libDF links, generated by `scripts/deepfilter-notices.py` after `build-deepfilter.sh`; rerun it when the DeepFilterNet pin changes | yes | no (not linked) | |
| 866 | |
| 867 | `LICENSE.webrtc` is WebRTC's own licence only. LiveKit's framework ships no |
| 868 | notices for the third-party code Google's WebRTC compiles in (libvpx, libyuv, |
| 869 | Opus, BoringSSL, abseil and others); those have not been gathered. |
| 870 | |
| 871 | ### Native interface |
| 872 | |
| 873 | See [DESIGN.md](DESIGN.md) for the first Oficina native visual direction, shared |
| 874 | views, screenshot workflow and remaining interaction work. |
| 875 | |
amadan