From 0cce3c97c2a6de9801501881f6d8200fd7d55b4c Mon Sep 17 00:00:00 2001 From: Kibi Kelburton Date: Thu, 1 Oct 2026 20:58:33 +0200 Subject: [PATCH 1/5] Add screen sharing between mumh5 users (picture only) - Signals travel as Mumble plugin data (compressed, chunked under Murmur's 1000 byte limit); the picture goes directly to each viewer over WebRTC - Toolbar button with a source picker in the desktop app, an indicator next to people who share, click to watch, a view above the chat with full screen - Late joiners learn about running streams; moving, stopping or disconnecting ends them for viewers - A note about IP addresses before the first use, optional STUN server setting - Unit tests for the signal codec and an E2E with two app instances Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + CLAUDE.md | 4 + README.md | 10 +- electron/main.ts | 38 ++++- electron/preload.ts | 2 + package.json | 1 + src/App.svelte | 4 +- src/core/share-signal.ts | 97 ++++++++++++ src/lib/actions.ts | 4 +- src/lib/icons.ts | 2 + src/lib/native.ts | 5 + src/lib/session.svelte.ts | 18 +++ src/lib/settings.svelte.ts | 4 + src/lib/share.svelte.ts | 288 +++++++++++++++++++++++++++++++++++ src/lib/ui.svelte.ts | 2 + src/ui/ChannelNode.svelte | 15 ++ src/ui/Chat.svelte | 2 + src/ui/SettingsDialog.svelte | 5 + src/ui/SharePicker.svelte | 35 +++++ src/ui/ShareView.svelte | 60 ++++++++ src/ui/Sidebar.svelte | 14 ++ test/e2e/share.e2e.ts | 107 +++++++++++++ test/share-signal.test.ts | 42 +++++ 23 files changed, 756 insertions(+), 4 deletions(-) create mode 100644 src/core/share-signal.ts create mode 100644 src/lib/share.svelte.ts create mode 100644 src/ui/SharePicker.svelte create mode 100644 src/ui/ShareView.svelte create mode 100644 test/e2e/share.e2e.ts create mode 100644 test/share-signal.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 543a0a8..0567791 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ All notable changes to mumh5. Versions follow the `version` in `package.json`. ### Added +- Screen sharing between mumh5 users in a channel (picture only so far): a toolbar button starts it, others click the indicator next to your name to watch. Streams go directly between clients; a STUN server can be set in Settings, Voice. - Browser version: `npm run build:web` builds the web app and a small self-hosted proxy that bridges browsers to Mumble servers on an allowlist. Identities are kept in the browser, voice goes through the TCP tunnel. See "Browser version" in the README. - Link previews in chat: title, description and image for web links (up to two per message). By default they are fetched by your f0ckm upload host, so the linked sites never see your IP address. Can be switched to "fetched by this computer" or off in Settings, Chat and files. - Voice tiles on small windows: when the window is too narrow for the member list, the people in your voice channel appear as tiles above the chat, light up while they talk, and keep mute and deafen at hand. The tiles can be collapsed. diff --git a/CLAUDE.md b/CLAUDE.md index 9759383..04efade 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,6 +26,7 @@ npm run build:web # browser build to dist-web/, proxy bundle to dist-proxy/ npm run dev:web # browser build with hot reload: starts the proxy (any server allowed) and Vite npm run proxy # run the proxy from source (MUMH5_SERVERS=host:port required) npm run test:e2e:web # drives the browser build through the proxy (needs npm run build:web first) +npm run test:e2e:share # screen sharing between two instances of the built app npm run proto # regenerate src/core/mumble-pb.js and mumble-udp-pb.js from proto/ ``` @@ -46,6 +47,7 @@ A reused test server keeps registrations and channels from earlier runs; tests m - `src/core/` (browser-safe TypeScript, also runs in Node for tests): framing and codec (`proto.ts`), the Mumble client state machine (`client.ts`), voice packet formats (`voice-packet.ts`). No DOM, no Electron, no Node imports here. - `src/lib/native.ts`: `desktop` is the Electron preload API or null; `native` is what both platforms provide (identities, certificates), backed by `web.svelte.ts` in the browser build (`isWeb`, vite `--mode web`). Desktop-only features check `desktop`. - `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets). `html.ts` sanitizes incoming HTML and serializes outgoing rich text. +- Screen sharing: `src/core/share-signal.ts` (signals as Mumble plugin data, id `mumh5.share`, compressed and chunked) and `src/lib/share.svelte.ts` (one WebRTC connection per viewer, candidates inside the description, no trickle). The desktop source picker is `share:sources` / `share:pick` in `electron/main.ts`. - `src/ui/`: Svelte components. `App.svelte` owns layout and global dialogs (`ui.svelte.ts` store). Voice runs on one server at a time (where you last joined a channel); background servers are auto self-deafened and restored when voice returns. @@ -62,6 +64,7 @@ Voice runs on one server at a time (where you last joined a channel); background - The CSP allows media only from `self`, `blob:` and http(s). `data:` audio is blocked silently: play user files through object URLs. Large user files go to IndexedDB (`src/lib/blobstore.ts`), not localStorage. - `window.prompt` does not exist in Electron; use `ui.prompt`. +- `desktopCapturer.getSources` can return an empty list on its first calls under X11; main retries. On Wayland the call itself opens the system picker, so it is never repeated. - A drop handler must read derived state before clearing the drag item it derives from. ## Svelte reactivity pitfalls (all hit before) @@ -76,6 +79,7 @@ Voice runs on one server at a time (where you last joined a channel); background - Murmur silently drops own UserState, TextMessage, ChannelState, ACL and Version past a leaky bucket (burst 5, 1/s). The client paces these; don't bypass `send()`. - 1.5 servers use the protobuf UDP voice format (type byte 0 + MumbleUDP.Audio) with clients announcing 1.5; older ones the legacy format. The UDPTunnel TCP body is the raw voice packet, not a protobuf message. Sequence numbers count 10 ms frames. - Long comments and descriptions arrive as a hash only; a new hash invalidates the old text; fetch with RequestBlob. +- Murmur drops PluginDataTransmission over 1000 bytes and rate limits it; keep packets at 900 bytes and few. - Renaming while connected is not possible; mumh5 reconnects with the new name. - Murmur never sends SuperUser (user id 0) PermissionQuery answers; treat SuperUser as allowed everything. - In ACLs, Write overrides denies. Grant test rights to one user (`$` group), not `@all`. diff --git a/README.md b/README.md index 0124713..9800877 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,14 @@ mumh5 keeps the foundation and replaces the experience. ![Profile with a formatted description](docs/screenshots/profile.png) +### Screen sharing + +- Share a screen or a window with the mumh5 users in your channel; they see an indicator next to your name and click it to watch +- No server setup and no extra account: the setup messages travel through the Mumble server, the picture goes directly between the two clients (WebRTC) +- Picture only for now, up to 8 viewers. Regular Mumble clients do not see streams +- Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Behind most home routers a STUN server (Settings, Voice) is needed; none is contacted unless you enter one +- Tested between two desktop instances on one machine; connections across the internet and the browser build are untested + ### Chat - Messages grouped by author, direct messages, unread badges - File sharing: drag and drop, paste, or the attach button, with upload progress @@ -254,7 +262,7 @@ Without an upload host, mumh5 still sends images, scaled to fit the server's lim - System-wide push to talk - Whisper and shout - Rich chat between mumh5 users: replies, reactions, edits, typing indicators -- Screen sharing between mumh5 users in a channel, with sound (one application or the whole system, also on Linux through PipeWire). Anyone can start a stream; no extra key, your Mumble certificate is your identity. Streams go directly between clients with WebRTC, set up through the Mumble server, with an optional self-hosted relay for many viewers +- Screen sharing: sound (one application or the whole system, also on Linux through PipeWire), quality settings, and an optional self-hosted relay for many viewers - Release builds for all platforms --- diff --git a/electron/main.ts b/electron/main.ts index f3aba65..c33439d 100644 --- a/electron/main.ts +++ b/electron/main.ts @@ -1,4 +1,4 @@ -import { app, BrowserWindow, dialog, ipcMain, net, shell, session, systemPreferences, type WebContents } from 'electron'; +import { app, BrowserWindow, desktopCapturer, dialog, ipcMain, net, shell, session, systemPreferences, type WebContents } from 'electron'; import { promises as fs } from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -12,6 +12,11 @@ import { udpChannel } from './udp-voice.ts'; const devUrl = process.env.VITE_DEV_SERVER_URL; +// Screen sharing connects mumh5 users directly. Chromium would hide this computer's addresses +// behind names that only resolve on the local network, so two people on IPv6 could not reach +// each other without a helper server. Only this app's own page runs here. +app.commandLine.appendSwitch('disable-features', 'WebRtcHideLocalIpsWithMdns'); + let identityStore: IdentityStore | null = null; const identities = () => (identityStore ??= new IdentityStore(app.getPath('userData'))); @@ -120,6 +125,30 @@ ipcMain.handle('publist:ping', (_e, host: string, port: number) => pingServer(St ipcMain.handle('preview:fetch', (_e, url: string) => fetchLinkPreview(String(url).slice(0, 2048))); +// ─── Screen sharing ─────────────────────────────────────────────────────────── +// The renderer lists the sources, the user picks one, and the next getDisplayMedia call gets it. +// On Wayland, listing already opens the system's picker and returns the one chosen source; +// `chosen` tells the renderer not to ask a second time. +let screenSources: Electron.DesktopCapturerSource[] = []; +let pickedSource: string | null = null; +ipcMain.handle('share:sources', async () => { + const wayland = process.platform === 'linux' && app.commandLine.getSwitchValue('ozone-platform') !== 'x11' && + (process.env.XDG_SESSION_TYPE === 'wayland' || !!process.env.WAYLAND_DISPLAY); + const list = () => desktopCapturer.getSources({ types: ['screen', 'window'], thumbnailSize: { width: 320, height: 180 } }); + screenSources = await list(); + // The X11 capturer can come back empty on its first calls. Not on Wayland, where empty + // means the user cancelled the system's picker and asking again would reopen it. + for (let i = 0; i < 8 && !screenSources.length && !wayland; i++) { + await new Promise(r => setTimeout(r, 250)); + screenSources = await list(); + } + return { + chosen: wayland && screenSources.length === 1, + sources: screenSources.map(s => ({ id: s.id, name: s.name, thumbnail: s.thumbnail.isEmpty() ? '' : s.thumbnail.toDataURL() })) + }; +}); +ipcMain.on('share:pick', (_e, id: string) => { pickedSource = String(id); }); + ipcMain.on('tray:update', (_e, state: tray.TrayState) => tray.update(state)); ipcMain.handle('platform:info', () => ({ os: process.platform, osVersion: os.release() })); @@ -195,6 +224,13 @@ app.whenReady().then(async () => { cb({ requestHeaders: details.requestHeaders }); }); + session.defaultSession.setDisplayMediaRequestHandler((_request, callback) => { + const source = screenSources.find(s => s.id === pickedSource); + pickedSource = null; + // Without a picked source the request is refused + try { callback(source ? { video: source } : {}); } catch { /* refused */ } + }); + // Microphone, camera and screen capture for voice and video; nothing else session.defaultSession.setPermissionRequestHandler((_wc, permission, cb) => { cb(['media', 'display-capture', 'clipboard-sanitized-write', 'notifications'].includes(permission)); diff --git a/electron/preload.ts b/electron/preload.ts index 704d66c..e8c2b76 100644 --- a/electron/preload.ts +++ b/electron/preload.ts @@ -21,6 +21,8 @@ contextBridge.exposeInMainWorld('mumh5Native', { onContextMenu: (fn: (params: unknown) => void) => { ipcRenderer.on('context-menu', (_e, p) => fn(p)); }, editAction: (action: string, arg?: unknown) => ipcRenderer.send('edit:action', action, arg), onTrayAction: (fn: (action: string) => void) => { ipcRenderer.on('tray:action', (_e, a: string) => fn(a)); }, + screenSources: () => ipcRenderer.invoke('share:sources'), + pickScreenSource: (id: string) => ipcRenderer.send('share:pick', id), describeCerts: (ders: Uint8Array[]) => ipcRenderer.invoke('certs:describe', ders), identities: { list: () => ipcRenderer.invoke('identities:list'), diff --git a/package.json b/package.json index 0504d72..78d7e2b 100644 --- a/package.json +++ b/package.json @@ -20,6 +20,7 @@ "dist:mac": "npm run build && electron-builder --mac", "test:e2e": "node test/e2e/app.e2e.ts", "test:e2e:web": "node test/e2e/web.e2e.ts", + "test:e2e:share": "node test/e2e/share.e2e.ts", "proto": "pbjs -t static-module -w es6 --keep-case --no-delimited --no-service --no-comments --force-number proto/Mumble.proto -o src/core/mumble-pb.js && pbjs -t static-module -w es6 --keep-case --no-delimited --no-service --no-comments --force-number proto/MumbleUDP.proto -o src/core/mumble-udp-pb.js", "screenshots": "node test/e2e/screenshots.ts" }, diff --git a/src/App.svelte b/src/App.svelte index 82cc44f..e39257b 100644 --- a/src/App.svelte +++ b/src/App.svelte @@ -15,6 +15,7 @@ import ChannelDialog from './ui/ChannelDialog.svelte'; import PublicServers from './ui/PublicServers.svelte'; import ServerInfoDialog from './ui/ServerInfoDialog.svelte'; + import SharePicker from './ui/SharePicker.svelte'; import ResizeHandle from './ui/ResizeHandle.svelte'; import { identities } from './lib/identities.svelte.ts'; import { session } from './lib/session.svelte.ts'; @@ -129,7 +130,8 @@ {#if ui.channelDialog && session.status === 'connected'} {#key ui.channelDialog} (ui.channelDialog = null)} />{/key} {/if} -{#if ui.prompt} (ui.prompt = null)} />{/if} + +{#if ui.prompt} { const p = ui.prompt; ui.prompt = null; p?.oncancel?.(); }} />{/if} diff --git a/src/ui/Chat.svelte b/src/ui/Chat.svelte index 4fc7532..ff9590a 100644 --- a/src/ui/Chat.svelte +++ b/src/ui/Chat.svelte @@ -3,6 +3,7 @@ import ChatLog from './ChatLog.svelte'; import Composer from './Composer.svelte'; import VoiceStage from './VoiceStage.svelte'; + import ShareView from './ShareView.svelte'; import { ui } from '../lib/ui.svelte.ts'; import { session } from '../lib/session.svelte.ts'; @@ -28,6 +29,7 @@ {#if showStage && session.status === 'connected'}{/if} + {#if session.status === 'connected'}{/if} {#if session.status === 'idle' && session.disconnectInfo} {@const d = session.disconnectInfo} diff --git a/src/ui/SettingsDialog.svelte b/src/ui/SettingsDialog.svelte index 1d30326..9c495f9 100644 --- a/src/ui/SettingsDialog.svelte +++ b/src/ui/SettingsDialog.svelte @@ -59,6 +59,10 @@ {#if tab === 'voice'} +

Screen sharing

+ + store.saveSettings()} placeholder="stun.example.com:3478" autocomplete="off" /> +

Screen sharing connects you directly to the people watching. Behind most home routers that needs a STUN server to find a route; it learns your IP address but never sees the stream. Empty means no outside server is contacted, which works on the same network and often over IPv6.

{:else if tab === 'sounds'} {:else if tab === 'appearance'} @@ -119,6 +123,7 @@ .tabs button { padding: 8px 14px; color: var(--text-dim); font-weight: 600; white-space: nowrap; } .tabs button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); } h3 { margin: 22px 0 0; font-size: 14px; } + .opt { text-transform: none; font-weight: 400; letter-spacing: 0; color: var(--text-faint); } .help { margin: 10px 0 0; font-size: 13px; color: var(--text-dim); } .row { display: flex; align-items: flex-end; gap: 8px; } .row > div { flex: 1; } diff --git a/src/ui/SharePicker.svelte b/src/ui/SharePicker.svelte new file mode 100644 index 0000000..7a39ff1 --- /dev/null +++ b/src/ui/SharePicker.svelte @@ -0,0 +1,35 @@ + + +{#if picker} + picker.choose(null)} width={720}> +

Pick a screen or a window. mumh5 users in your channel can then watch it.

+
    + {#each picker.sources as s (s.id)} +
  • + +
    • + {/each} +
+ {#snippet footer()} + + {/snippet} + +{/if} + + diff --git a/src/ui/ShareView.svelte b/src/ui/ShareView.svelte new file mode 100644 index 0000000..e9dc267 --- /dev/null +++ b/src/ui/ShareView.svelte @@ -0,0 +1,60 @@ + + +{#if mine} +
+ + You are sharing your screen{share.viewers ? `, ${share.viewers} watching` : ''} + +
+{/if} +{#if share.error}{/if} + +{#if watching} +
+
+ + {sharer ? store.displayName(sharer) : 'Screen share'} + + +
+
+ + + {#if watching.state === 'connecting'}

Connecting...

+ {:else if watching.state === 'failed'}

Could not connect. You and the other person may both be behind routers that block direct connections; a STUN server in Settings, Voice can help.

{/if} +
+
+{/if} + + diff --git a/src/ui/Sidebar.svelte b/src/ui/Sidebar.svelte index 6ccfb94..dbef956 100644 --- a/src/ui/Sidebar.svelte +++ b/src/ui/Sidebar.svelte @@ -9,6 +9,7 @@ import { PERM } from '../core/client.ts'; import ResizeHandle from './ResizeHandle.svelte'; import { voice } from '../lib/audio/voice.svelte.ts'; + import { share } from '../lib/share.svelte.ts'; // resizable: false in the phone drawer, which keeps a fixed width // handleEdge: which side the resize handle sits on (left when the list is on the right, classic layout) @@ -50,6 +51,14 @@ ui.panelOpen = true; } + // Sharing runs on one server at a time; the button stops it from anywhere + const sharingHere = $derived(!!share.stream); + const canShare = $derived(session.status === 'connected' && share.supported(sessions.active)); + function toggleShare() { + if (share.stream) share.stop(); + else share.start(sessions.active); + } + // One button for the whole tree: open everything, then close everything let allOpen = $state(false); function toggleAll() { @@ -159,6 +168,10 @@ onclick={() => session.setSelfDeaf(!deafened)} oncontextmenu={e => openQuick(e, 'output')}> + - {#if showStage && session.status === 'connected'}{/if} - {#if session.status === 'connected'}{/if} + {#if session.status === 'connected'}{/if} {#if session.status === 'idle' && session.disconnectInfo} {@const d = session.disconnectInfo} diff --git a/src/ui/RightPanel.svelte b/src/ui/RightPanel.svelte index 61431a1..f8de5b0 100644 --- a/src/ui/RightPanel.svelte +++ b/src/ui/RightPanel.svelte @@ -13,6 +13,8 @@ import { voice } from '../lib/audio/voice.svelte.ts'; import { renderIncoming } from '../lib/html.ts'; import RichEditor from './RichEditor.svelte'; + import { share } from '../lib/share.svelte.ts'; + import { sessions } from '../lib/session.svelte.ts'; let { onnavigate, onclose }: { onnavigate: () => void; onclose: () => void } = $props(); @@ -124,6 +126,16 @@ + {#if session.sharing[user.session]} + + {/if} + {#if isSelf}
{/if} @@ -158,6 +170,9 @@ diff --git a/src/ui/SharePicker.svelte b/src/ui/SharePicker.svelte deleted file mode 100644 index 7a39ff1..0000000 --- a/src/ui/SharePicker.svelte +++ /dev/null @@ -1,35 +0,0 @@ - - -{#if picker} - picker.choose(null)} width={720}> -

Pick a screen or a window. mumh5 users in your channel can then watch it.

-
    - {#each picker.sources as s (s.id)} -
  • - -
    • - {/each} -
- {#snippet footer()} - - {/snippet} - -{/if} - - diff --git a/src/ui/ShareView.svelte b/src/ui/ShareView.svelte deleted file mode 100644 index e9dc267..0000000 --- a/src/ui/ShareView.svelte +++ /dev/null @@ -1,60 +0,0 @@ - - -{#if mine} -
- - You are sharing your screen{share.viewers ? `, ${share.viewers} watching` : ''} - -
-{/if} -{#if share.error}{/if} - -{#if watching} -
-
- - {sharer ? store.displayName(sharer) : 'Screen share'} - - -
-
- - - {#if watching.state === 'connecting'}

Connecting...

- {:else if watching.state === 'failed'}

Could not connect. You and the other person may both be behind routers that block direct connections; a STUN server in Settings, Voice can help.

{/if} -
-
-{/if} - - diff --git a/src/ui/VoiceStage.svelte b/src/ui/VoiceStage.svelte index d489011..8fab3d2 100644 --- a/src/ui/VoiceStage.svelte +++ b/src/ui/VoiceStage.svelte @@ -7,11 +7,45 @@ import { ui } from '../lib/ui.svelte.ts'; import { userMenu } from '../lib/actions.ts'; import { voice } from '../lib/audio/voice.svelte.ts'; + import { share } from '../lib/share.svelte.ts'; + import { sessions } from '../lib/session.svelte.ts'; + + // People in your voice channel as tiles. Shown on windows too narrow for the member list + // (always), and on any window while someone in the channel shares a screen: a sharer's tile + // carries the picture, and clicking it puts it on a large stage with the tiles as a strip. + let { always = false }: { always?: boolean } = $props(); - // People in your voice channel as tiles, for windows too narrow for the member list const self = $derived(session.self); const channel = $derived(self ? session.channel(self.channelId) : undefined); const people = $derived(self ? session.usersIn(self.channelId) : []); + const here = (s: { server?: { id: string } | null } | null | undefined) => !!s && s.server?.id === session.server?.id; + + const watching = $derived(share.watching && here(share.watching.host) ? share.watching : null); + const mine = $derived(share.stream && here(share.host) ? share.stream : null); + const anySharing = $derived(people.some(u => session.sharing[u.session])); + const visible = $derived(!!self && (always || anySharing || !!watching || !!mine)); + + // The picture a tile can show: our own stream, or the one we are watching + function streamOf(s: number): MediaStream | null { + if (s === self?.session) return mine; + return watching?.session === s ? watching.stream : null; + } + + // Tile on the large stage + let focus = $state(null); + const focused = $derived(focus != null ? streamOf(focus) : null); + const focusUser = $derived(focus != null ? session.user(focus) : undefined); + // A stream we just started watching goes to the stage; it leaves when the stream ends + let lastWatch: number | null = null; + $effect(() => { + const now = watching?.session ?? null; + if (now !== lastWatch) { + if (now != null) focus = now; + else if (focus === lastWatch) focus = null; + lastWatch = now; + } + }); + $effect(() => { if (focus === self?.session && !mine) focus = null; }); let collapsed = $state(load()); function load(): boolean { @@ -23,10 +57,39 @@ } const talking = (s: number) => !!voice.talking[s] || (s === self?.session && voice.transmitting); + + // The same stream can sit in a tile and on the stage at once + function media(node: HTMLMediaElement, stream: MediaStream | null) { + node.srcObject = stream; + return { update(next: MediaStream | null) { if (node.srcObject !== next) node.srcObject = next; } }; + } + + // Sound comes from one hidden player, so it plays once however many pictures are shown + let player = $state(); + const hasSound = $derived(!!watching?.stream?.getAudioTracks().length); + $effect(() => { + if (!player) return; + player.volume = store.settings.shareVolume; + player.muted = store.settings.shareVolume === 0; + }); + + function tileClick(session_: number) { + if (streamOf(session_)) focus = focus === session_ ? null : session_; + else if (session.sharing[session_] && session_ !== self?.session) share.watch(sessions.active, session_); + else { session.showUser(session_); ui.panelOpen = true; } + } + + let frame = $state(); + function fullscreen() { + if (document.fullscreenElement) document.exitFullscreen(); + else frame?.requestFullscreen().catch(() => {}); + } -{#if self} -
+{#if share.error}{/if} + +{#if visible && self} +
+ {#if mine} + You are sharing your screen{mine.getAudioTracks().length ? ' with sound' : ''}{share.viewers ? `, ${share.viewers} watching` : ''} + + {/if}
+ + {#if watching?.stream}{/if} + + {#if focused && focus != null} +
+
+ + {focus === self.session ? 'Your screen' : focusUser ? store.displayName(focusUser) : 'Screen share'} + {#if focus !== self.session && hasSound} + + store.saveSettings()} + aria-label="Stream volume" title="Stream volume" /> + {/if} + + {#if focus !== self.session}{/if} + +
+ + +
+ {/if} + {#if !collapsed} -
    +
      {#each people as u (u.session)} -
    • -
      • {/each}
    + {#if watching?.state === 'failed'} +

    Could not connect to the stream. You and the other person may both be behind routers that block direct connections; a STUN server in Settings, Voice can help.

    + {/if} {/if}
{/if} @@ -89,4 +195,28 @@ .tile.self .who { color: var(--text); font-weight: 600; } .badge { position: absolute; top: 4px; right: 4px; display: inline-flex; color: var(--text-faint); } .badge.admin { color: var(--danger); } + .badge.live { color: var(--speaking); } + + /* Streams */ + .bar { flex: none; display: flex; align-items: center; gap: 8px; padding: 6px 12px; font-size: 13px; background: var(--bg-1); border-bottom: 1px solid var(--line); } + .bar span { flex: 1; min-width: 0; } + .bar.bad { color: var(--danger); } + .btn.small, .bar .btn { padding: 4px 10px; font-size: 12px; flex: none; } + .status { font-size: 12px; color: var(--speaking); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; min-width: 0; margin-right: 4px; } + .tiles li.wide { grid-column: span 2; } + .tile.video { padding: 0 0 6px; overflow: hidden; } + .tile.video video { width: 100%; aspect-ratio: 16 / 9; object-fit: contain; background: #000; display: block; } + .tile.focused { border-color: var(--accent); box-shadow: inset 0 0 0 1px var(--accent); } + .note { font-size: 11px; color: var(--text-faint); } + .failed { margin: 0; padding: 0 12px 10px; font-size: 12px; color: var(--danger); } + /* Meeting view: the chosen picture large, everyone else in one row below */ + .stage.meeting { display: flex; flex-direction: column; max-height: 72%; min-height: 220px; } + .spot { flex: 1; min-height: 0; display: flex; flex-direction: column; background: #000; } + .spot-head { flex: none; display: flex; align-items: center; gap: 8px; padding: 2px 6px 2px 12px; background: var(--bg-0); color: var(--text-dim); font-size: 13px; } + .spot-who { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--text); font-weight: 600; } + .spot video { flex: 1; min-height: 0; width: 100%; object-fit: contain; background: #000; display: block; } + .vol { width: 110px; flex: none; padding: 0; } + .tiles.strip { flex: none; display: flex; overflow-x: auto; overflow-y: hidden; padding-top: 8px; max-height: none; } + .tiles.strip li { flex: 0 0 96px; } + .tiles.strip li.wide { flex-basis: 150px; } diff --git a/test/e2e/app.e2e.ts b/test/e2e/app.e2e.ts index 06e8866..0889fa3 100644 --- a/test/e2e/app.e2e.ts +++ b/test/e2e/app.e2e.ts @@ -886,6 +886,8 @@ try { // Custom icon from the tile's context menu: square-cropped and shown instead of initials const iconFile = path.join(userData, 'icon.png'); writeFileSync(iconFile, testPng(300, 200)); + // With hints off, the title of whatever the pointer rests on is set aside; move off the tile first + await page.mouse.move(700, 400); await page.locator('.rail .tile[title^="Second"]').click({ button: 'right' }); const chooser = page.waitForEvent('filechooser'); await page.getByRole('menuitem', { name: 'Change icon...' }).click(); diff --git a/test/e2e/share.e2e.ts b/test/e2e/share.e2e.ts index f4246bf..3956cee 100644 --- a/test/e2e/share.e2e.ts +++ b/test/e2e/share.e2e.ts @@ -56,45 +56,83 @@ try { await shareBtn.click(); await alice.page.getByText(/You see each other's IP address/).waitFor(); await alice.page.getByRole('button', { name: 'Continue' }).click(); + // The dialog: pick a source, see the preview, then start const picker = alice.page.getByRole('dialog', { name: 'Share your screen' }); - await picker.locator('.sources button').first().click(); + const startBtn = picker.getByRole('button', { name: 'Start sharing' }); + const pickAndStart = async () => { + assert.ok(await startBtn.isDisabled(), 'nothing can start before a source is chosen'); + // Listing is slow on the bare test display and can come back empty the first time + for (let i = 0; i < 6 && !(await picker.locator('.sources button').count()); i++) { + await Promise.race([picker.locator('.sources button').first().waitFor({ timeout: 40000 }), picker.getByText('Nothing found').waitFor({ timeout: 40000 })]).catch(() => {}); + if (await picker.getByText('Nothing found').count()) await picker.getByRole('button', { name: 'Refresh' }).click(); + } + await picker.locator('.sources button').first().click(); + await alice.page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 }); + await startBtn.click(); + }; + await picker.getByLabel('Quality').selectOption('low'); + await pickAndStart(); await alice.page.getByText('You are sharing your screen').waitFor(); await alice.page.locator('.sidebar .row.user.self').getByTitle('You are sharing your screen').waitFor(); - console.log('ok: sharing started from the picker'); + // The sharer's own tile shows what is being sent; clicking it makes it large + await alice.page.waitForFunction(() => (document.querySelector('.stage .tile.self video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 }); + await alice.page.locator('.stage .tile.self').click(); + const ownView = alice.page.getByRole('region', { name: 'Screen share' }); + await ownView.getByText('Your screen').waitFor(); + await ownView.getByRole('button', { name: 'Back to tiles' }).click(); + await ownView.waitFor({ state: 'detached' }); + console.log('ok: sharing started from the dialog, own tile shows the stream'); // Bob connects afterwards and still learns about the stream await join(bob.page, bobName); - const watch = bob.page.getByRole("button", { name: `Watch the screen of ${aliceName}`, exact: true }); + // Next to the name in the channel list, and on the tile above the chat + const watch = bob.page.locator('.sidebar').getByRole('button', { name: `Watch the screen of ${aliceName}`, exact: true }); await watch.waitFor(); + const tile = bob.page.locator('.stage .tile', { hasText: aliceName }); + await tile.waitFor(); console.log('ok: a late joiner sees who is sharing'); - await watch.click(); + await tile.click(); await bob.page.getByRole('button', { name: 'Continue' }).click(); const view = bob.page.getByRole('region', { name: 'Screen share' }); await view.getByText(aliceName).waitFor(); - await bob.page.waitForFunction(() => (document.querySelector('.watch video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 }); - const size = await bob.page.evaluate(() => { const v = document.querySelector('.watch video') as HTMLVideoElement; return [v.videoWidth, v.videoHeight]; }); + await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 }); + const size = await bob.page.evaluate(() => { const v = document.querySelector('.spot video') as HTMLVideoElement; return [v.videoWidth, v.videoHeight]; }); await alice.page.getByText('You are sharing your screen, 1 watching').waitFor(); console.log(`ok: bob receives the picture (${size[0]}x${size[1]}), alice sees one viewer`); + // Like a meeting: back to tiles keeps the picture in the tile, a click makes it large again + await view.getByRole('button', { name: 'Back to tiles' }).click(); + await view.waitFor({ state: 'detached' }); + await bob.page.waitForFunction(() => (document.querySelector('.stage .tile.video video') as HTMLVideoElement | null)?.videoWidth! > 0); + await tile.click(); + await view.waitFor(); + console.log('ok: stream in the tile, large on click'); + // Leaving and coming back await view.getByRole('button', { name: 'Stop watching' }).click(); await view.waitFor({ state: 'detached' }); await alice.page.getByText('You are sharing your screen', { exact: true }).waitFor(); - await watch.click(); - await bob.page.waitForFunction(() => (document.querySelector('.watch video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 }); - console.log('ok: stop watching and watch again'); + // The sharer's profile says so and offers to watch + await bob.page.locator('.sidebar .row.user', { hasText: aliceName }).click(); + const profile = bob.page.locator('.panel .sharing'); + await profile.getByText('Sharing their screen').waitFor(); + await profile.getByRole('button', { name: 'Watch' }).click(); + await profile.getByRole('button', { name: 'Stop watching' }).waitFor(); + await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 }); + console.log('ok: stop watching, watch again from the profile'); // Alice stops: the view and the indicator go away await alice.page.locator('.me').getByRole('button', { name: 'Stop sharing your screen' }).click(); await view.waitFor({ state: 'detached' }); await watch.waitFor({ state: 'detached' }); + await profile.waitFor({ state: 'detached' }); assert.equal(await alice.page.getByText('You are sharing your screen').count(), 0); console.log('ok: stopping ends the stream for viewers'); // A sharer who disconnects takes the stream with them await shareBtn.click(); - await picker.locator('.sources button').first().click(); + await pickAndStart(); await watch.waitFor(); await alice.page.getByTitle('Disconnect').click(); await watch.waitFor({ state: 'detached' }); diff --git a/test/pipewire.test.ts b/test/pipewire.test.ts new file mode 100644 index 0000000..c83ed2a --- /dev/null +++ b/test/pipewire.test.ts @@ -0,0 +1,47 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { audioApps, planLinks, VIRTUAL_MIC } from '../electron/pipewire.ts'; + +// A cut-down pw-dump: the virtual microphone (10), this app (20, process 500), a browser whose +// stream has no name of its own (30, named by its client), a mono game (40) and a music player (50) +const node = (id: number, props: Record) => ({ id, type: 'PipeWire:Interface:Node', info: { props } }); +const client = (id: number, name: string, pid: number) => ({ id, type: 'PipeWire:Interface:Client', info: { props: { 'application.name': name, 'application.process.id': pid } } }); +const port = (id: number, nodeId: number, direction: string, channel: string) => ({ id, type: 'PipeWire:Interface:Port', info: { direction, props: { 'node.id': nodeId, 'audio.channel': channel } } }); +const link = (id: number, outPort: number, inPort: number, inNode: number) => + ({ id, type: 'PipeWire:Interface:Link', info: { 'output-port-id': outPort, 'input-port-id': inPort, 'input-node-id': inNode } as any }); + +const graph = [ + node(10, { 'node.name': VIRTUAL_MIC, 'media.class': 'Audio/Source/Virtual' }), port(11, 10, 'input', 'FL'), port(12, 10, 'input', 'FR'), + port(13, 10, 'output', 'FL'), port(14, 10, 'output', 'FR'), + client(2, 'mumh5', 500), node(20, { 'media.class': 'Stream/Output/Audio', 'node.name': 'mumh5', 'client.id': 2 }), port(21, 20, 'output', 'FL'), port(22, 20, 'output', 'FR'), + client(3, 'Firefox', 600), node(30, { 'media.class': 'Stream/Output/Audio', 'node.name': 'Firefox', 'client.id': 3 }), port(31, 30, 'output', 'FL'), port(32, 30, 'output', 'FR'), + node(40, { 'media.class': 'Stream/Output/Audio', 'application.name': 'Game', 'application.process.id': 700 }), port(41, 40, 'output', 'MONO'), + node(50, { 'media.class': 'Stream/Output/Audio', 'application.name': 'Spotify' }), port(51, 50, 'output', 'FL'), port(52, 50, 'output', 'FR'), port(53, 50, 'output', 'LFE'), + node(60, { 'media.class': 'Audio/Sink', 'node.name': 'speakers' }) +]; + +test('lists programs that play sound, without this app', () => { + assert.deepEqual(audioApps(graph, [500]), ['Firefox', 'Game', 'Spotify']); +}); + +test('whole system links every program but this app', () => { + const { add, remove } = planLinks(graph, { kind: 'all' }, [500]); + assert.deepEqual(add.sort(), [[31, 11], [32, 12], [41, 11], [41, 12], [51, 11], [52, 12]].sort()); + assert.deepEqual(remove, []); +}); + +test('one program links only that program', () => { + assert.deepEqual(planLinks(graph, { kind: 'app', name: 'Firefox' }, [500]).add, [[31, 11], [32, 12]]); +}); + +test('existing links are kept, stale ones removed', () => { + const linked = [...graph, link(90, 31, 11, 10), link(91, 51, 11, 10), link(92, 21, 60, 60)]; + const { add, remove } = planLinks(linked, { kind: 'app', name: 'Firefox' }, [500]); + assert.deepEqual(add, [[32, 12]]); + // Spotify's link goes; the link to the speakers is none of our business + assert.deepEqual(remove, [91]); +}); + +test('nothing without the virtual microphone', () => { + assert.deepEqual(planLinks(graph.slice(5), { kind: 'all' }, []), { add: [], remove: [] }); +}); diff --git a/test/proxy.test.ts b/test/proxy.test.ts index 68f1ec5..4ed867f 100644 --- a/test/proxy.test.ts +++ b/test/proxy.test.ts @@ -181,7 +181,8 @@ test('two clients talk through the proxy', { skip: !target }, async () => { const a = await connect(`proxy-a-${Date.now() % 100000}`); const b = await connect(`proxy-b-${Date.now() % 100000}`); const got = new Promise(resolve => b.on('text', m => resolve(m.html))); - a.sendText({ channels: [0] }, 'hello through the proxy'); + // A direct message, so other test files listening in the root channel are not disturbed + a.sendText({ users: [b.session!] }, 'hello through the proxy'); assert.equal(await got, 'hello through the proxy'); // No UDP in a browser: voice stays on the TCP tunnel assert.equal(a.udpOk, false); From 13b7013004544b1a4613f7a7572751da1eef45e9 Mon Sep 17 00:00:00 2001 From: Kibi Kelburton Date: Thu, 1 Oct 2026 21:30:17 +0200 Subject: [PATCH 3/5] Say that both sides need a STUN server for streams across the internet Co-Authored-By: Claude Opus 5.5 --- src/ui/SettingsDialog.svelte | 2 +- src/ui/VoiceStage.svelte | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ui/SettingsDialog.svelte b/src/ui/SettingsDialog.svelte index 9c495f9..5c25220 100644 --- a/src/ui/SettingsDialog.svelte +++ b/src/ui/SettingsDialog.svelte @@ -62,7 +62,7 @@

Screen sharing

store.saveSettings()} placeholder="stun.example.com:3478" autocomplete="off" /> -

Screen sharing connects you directly to the people watching. Behind most home routers that needs a STUN server to find a route; it learns your IP address but never sees the stream. Empty means no outside server is contacted, which works on the same network and often over IPv6.

+

Screen sharing connects you directly to the people watching. Across the internet both the person sharing and the people watching need a STUN server here to find a route; it learns your IP address but never sees the stream. Empty means no outside server is contacted, which only works on the same network.

{:else if tab === 'sounds'} {:else if tab === 'appearance'} diff --git a/src/ui/VoiceStage.svelte b/src/ui/VoiceStage.svelte index 8fab3d2..440a25d 100644 --- a/src/ui/VoiceStage.svelte +++ b/src/ui/VoiceStage.svelte @@ -169,7 +169,7 @@ {/each} {#if watching?.state === 'failed'} -

Could not connect to the stream. You and the other person may both be behind routers that block direct connections; a STUN server in Settings, Voice can help.

+

Could not connect to the stream. Across the internet both of you need a STUN server set in Settings, Voice (the person sharing has to restart the stream after setting it).

{/if} {/if}
From 826542ed8af0ecbbb0d62f5422d36c840cf93cde Mon Sep 17 00:00:00 2001 From: Kibi Kelburton Date: Thu, 1 Oct 2026 21:38:42 +0200 Subject: [PATCH 4/5] Add a STUN responder to the proxy for screen sharing between browsers The browser build uses it automatically, so no outside server is contacted. Covered by unit tests and a two-browser step in the web E2E. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 2 +- README.md | 3 +- scripts/dev-web.mjs | 2 +- server/main.ts | 3 +- server/proxy.ts | 63 +++++++++++++++++++++++++++++++++++++++-- src/lib/share.svelte.ts | 8 ++++-- src/lib/web.svelte.ts | 11 ++++++- test/e2e/web-shell.cjs | 8 +++++- test/e2e/web.e2e.ts | 35 +++++++++++++++++++++-- test/proxy.test.ts | 45 ++++++++++++++++++++++++++++- 10 files changed, 166 insertions(+), 14 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 2b1ba21..4f79a1a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,7 @@ A reused test server keeps registrations and channels from earlier runs; tests m ## Architecture - `electron/` (Node, main process): window, TLS sockets to Mumble servers (`tls-transport.ts`), encrypted UDP voice (`udp-voice.ts`, `ocb2.ts`, tested against Mumble's OCB2 vectors), identities and PKCS#12 (`identity.ts`, `identity-store.ts`), certificate parsing (`certs.ts`), tray (`tray.ts`). The renderer only gets the narrow `window.mumh5Native` API from `preload.ts` (context isolation, sandbox). -- `server/` (Node): the web proxy. `proxy.ts` serves `dist-web`, bridges WebSocket connections to Mumble over TLS (reusing `electron/tls-transport.ts`) and has stateless identity endpoints. It stores nothing; the browser keeps identities in localStorage and sends one with each connect. +- `server/` (Node): the web proxy. `proxy.ts` serves `dist-web`, bridges WebSocket connections to Mumble over TLS (reusing `electron/tls-transport.ts`) and has stateless identity endpoints. It stores nothing; the browser keeps identities in localStorage and sends one with each connect. It also answers STUN on UDP for screen sharing between browser users. - `src/core/` (browser-safe TypeScript, also runs in Node for tests): framing and codec (`proto.ts`), the Mumble client state machine (`client.ts`), voice packet formats (`voice-packet.ts`). No DOM, no Electron, no Node imports here. - `src/lib/native.ts`: `desktop` is the Electron preload API or null; `native` is what both platforms provide (identities, certificates), backed by `web.svelte.ts` in the browser build (`isWeb`, vite `--mode web`). Desktop-only features check `desktop`. - `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets). `html.ts` sanitizes incoming HTML and serializes outgoing rich text. diff --git a/README.md b/README.md index 4a820df..d827394 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ mumh5 keeps the foundation and replaces the experience. - Sound: on Linux one program or everything except mumh5 itself (through PipeWire, so viewers do not hear the voice chat twice); on Windows the whole system; in a browser what the browser offers - People in the channel see an indicator next to your name, on your tile and in your profile, and click to watch. Your tile shows the picture; a click makes it large with everyone else in a strip below, like a meeting. Viewers set the stream's volume - No server setup and no extra account: the setup messages travel through the Mumble server, the stream goes directly between the two clients (WebRTC), up to 8 viewers. Regular Mumble clients do not see streams -- Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Behind most home routers a STUN server (Settings, Voice) is needed; none is contacted unless you enter one +- Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Across the internet both sides need a STUN server. The browser version uses the one built into its proxy; in the desktop app you enter one in Settings, Voice (the proxy's address works there too), and none is contacted unless you do - Tested between two desktop instances on one machine with a test picture. Sound capture, real screens, connections across the internet, Windows and the browser build are untested ### Chat @@ -176,6 +176,7 @@ Then open `http://127.0.0.1:8080`. For development, `npm run dev:web` starts the | `MUMH5_ORIGINS` | same host | Origins allowed to use the API, comma-separated, when the page is hosted elsewhere | | `MUMH5_TRUST_PROXY` | off | Take client addresses from `X-Forwarded-For` (set this behind a reverse proxy) | | `MUMH5_SEND_PROXY` | off | Announce each visitor's address to the server with the PROXY protocol (see below). Breaks connections to a plain Mumble server | +| `MUMH5_STUN_PORT`, `MUMH5_STUN_BIND` | `3478`, all addresses | UDP port of the built-in STUN responder that lets browser users find a direct route for screen sharing. Open this UDP port in the firewall; it does not go through nginx. `0` turns it off | | `MUMH5_STATIC` | `../dist-web` | Folder with the web build | | `MUMH5_MAX_CONNECTIONS`, `MUMH5_MAX_PER_ADDRESS` | `200`, `8` | Connection limits, in total and per client address | diff --git a/scripts/dev-web.mjs b/scripts/dev-web.mjs index a1f599b..268abd5 100644 --- a/scripts/dev-web.mjs +++ b/scripts/dev-web.mjs @@ -4,7 +4,7 @@ import { createServer } from 'vite'; // The browser build for development: the web proxy on 127.0.0.1:8080 and the Vite dev server, // which forwards /api to it (vite.config.ts). Without MUMH5_SERVERS the proxy allows any // server, private addresses included; it only listens on this machine. -const env = { ...process.env, MUMH5_PORT: '8080', MUMH5_BIND: '127.0.0.1' }; +const env = { MUMH5_STUN_BIND: '127.0.0.1', ...process.env, MUMH5_PORT: '8080', MUMH5_BIND: '127.0.0.1' }; if (!env.MUMH5_SERVERS && !env.MUMH5_ALLOW_ANY) Object.assign(env, { MUMH5_ALLOW_ANY: '1', MUMH5_ALLOW_PRIVATE: '1' }); const proxy = spawn(process.execPath, ['server/main.ts'], { stdio: 'inherit', env }); diff --git a/server/main.ts b/server/main.ts index ddde361..d0ea103 100644 --- a/server/main.ts +++ b/server/main.ts @@ -8,10 +8,11 @@ const config = configFromEnv(process.env); config.staticDir ??= [path.resolve(import.meta.dirname, '../dist-web')].find(d => existsSync(path.join(d, 'index.html'))) ?? null; try { - const { port } = await startProxy(config); + const { port, stunPort } = await startProxy(config); console.log(`mumh5 proxy listening on http://${config.bind}:${port}`); console.log(config.allowAny ? `Allowed servers: any${config.allowPrivate ? ', private addresses included' : ' public address'}` : `Allowed servers: ${config.servers.map(s => `${s.host}:${s.port}`).join(', ')}`); if (config.sendProxy) console.log('Announcing client addresses with the PROXY protocol; the allowed servers must expect it'); + console.log(stunPort ? `STUN for screen sharing on UDP port ${stunPort} (must be reachable from the internet)` : 'STUN is off; screen sharing between browser users will only work on the same network'); console.log(config.staticDir ? `Serving the web app from ${config.staticDir}` : 'No web build found (npm run build:web); serving the API only'); } catch (e) { console.error((e as Error).message); diff --git a/server/proxy.ts b/server/proxy.ts index 1e872ae..b820411 100644 --- a/server/proxy.ts +++ b/server/proxy.ts @@ -4,6 +4,7 @@ import http from 'node:http'; import dns from 'node:dns'; import net from 'node:net'; +import dgram from 'node:dgram'; import path from 'node:path'; import { promises as fs } from 'node:fs'; import { WebSocketServer, type WebSocket } from 'ws'; @@ -29,13 +30,16 @@ export interface ProxyConfig { // Only for servers behind something that understands it (go-mmproxy); plain Mumble does not. sendProxy: boolean; staticDir: string | null; + // UDP port of the built-in STUN responder for screen sharing between browser users; null turns it off + stunPort: number | null; + stunBind: string; maxConnections: number; maxPerAddress: number; } export const defaults: ProxyConfig = { port: 8080, bind: '127.0.0.1', servers: [], allowAny: false, allowPrivate: false, origins: [], - trustProxy: false, sendProxy: false, staticDir: null, maxConnections: 200, maxPerAddress: 8 + trustProxy: false, sendProxy: false, staticDir: null, stunPort: null, stunBind: '::', maxConnections: 200, maxPerAddress: 8 }; // "host", "host:port", "[v6]:port", each optionally followed by "=Label" @@ -64,6 +68,8 @@ export function configFromEnv(env: NodeJS.ProcessEnv): ProxyConfig { trustProxy: on(env.MUMH5_TRUST_PROXY), sendProxy: on(env.MUMH5_SEND_PROXY), staticDir: env.MUMH5_STATIC ?? null, + stunPort: Number(env.MUMH5_STUN_PORT ?? 3478) || null, + stunBind: env.MUMH5_STUN_BIND ?? defaults.stunBind, maxConnections: Number(env.MUMH5_MAX_CONNECTIONS ?? defaults.maxConnections), maxPerAddress: Number(env.MUMH5_MAX_PER_ADDRESS ?? defaults.maxPerAddress) }; @@ -93,6 +99,39 @@ const publicLookup: net.LookupFunction = (hostname, options, callback) => { }); }; +// Answer to a STUN binding request (RFC 5389): tells the sender the address its packet came +// from, which is how two browsers behind routers find a direct route for screen sharing. +// Returns null for anything that is not a binding request. The answer is about as small as +// the request, so the port is of no use for amplifying traffic. +export function stunResponse(msg: Uint8Array, address: string, port: number): Uint8Array | null { + const COOKIE = 0x2112a442; + const view = new DataView(msg.buffer, msg.byteOffset, msg.byteLength); + if (msg.length < 20 || view.getUint16(0) !== 0x0001 || view.getUint32(4) !== COOKIE) return null; + if (view.getUint16(2) !== msg.length - 20) return null; + const v4 = address.replace(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/i, '$1'); + let bytes: number[]; + if (net.isIPv4(v4)) bytes = v4.split('.').map(Number); + else if (net.isIPv6(address)) { + // Expand "::" and write the eight groups out as bytes + const [head, tail = ''] = address.split('%')[0].split('::'); + const h = head ? head.split(':') : [], t = tail ? tail.split(':') : []; + const groups = address.includes('::') ? [...h, ...new Array(8 - h.length - t.length).fill('0'), ...t] : h; + bytes = groups.flatMap(g => { const n = parseInt(g, 16); return [n >> 8, n & 255]; }); + } else return null; + const out = new Uint8Array(20 + 8 + bytes.length); + const o = new DataView(out.buffer); + o.setUint16(0, 0x0101); // binding success + o.setUint16(2, 8 + bytes.length); + out.set(msg.subarray(4, 20), 4); // cookie and transaction id + o.setUint16(20, 0x0020); // XOR-MAPPED-ADDRESS + o.setUint16(22, 4 + bytes.length); + out[25] = bytes.length === 4 ? 1 : 2; + o.setUint16(26, port ^ (COOKIE >>> 16)); + // The address is masked with the cookie, and for IPv6 with the transaction id after it + for (let i = 0; i < bytes.length; i++) out[28 + i] = bytes[i] ^ msg[4 + i]; + return out; +} + const TYPES: Record = { '.html': 'text/html; charset=utf-8', '.js': 'text/javascript; charset=utf-8', '.css': 'text/css; charset=utf-8', '.json': 'application/json', '.png': 'image/png', '.svg': 'image/svg+xml', '.ico': 'image/x-icon', '.jpg': 'image/jpeg', @@ -110,12 +149,13 @@ class HttpError extends Error { constructor(status: number, message: string) { super(message); this.status = status; } } -export async function startProxy(config: ProxyConfig): Promise<{ port: number; close(): Promise }> { +export async function startProxy(config: ProxyConfig): Promise<{ port: number; stunPort: number | null; close(): Promise }> { if (!config.allowAny && !config.servers.length) { throw new Error('No servers allowed. Set MUMH5_SERVERS=host[:port][=Label],... or MUMH5_ALLOW_ANY=1.'); } const staticDir = config.staticDir ? path.resolve(config.staticDir) : null; const perAddress = new Map(); + let stunPort: number | null = null; // Identity requests per address in the current minute; key generation is the costly part const identityUse = new Map(); const sweep = setInterval(() => identityUse.clear(), 60000); @@ -154,7 +194,7 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; c async function api(req: http.IncomingMessage, route: string): Promise { if (route === 'config' && req.method === 'GET') { - return { servers: config.servers, any: config.allowAny }; + return { servers: config.servers, any: config.allowAny, stun: stunPort }; } if (req.method !== 'POST') throw new HttpError(404, 'Not found'); if (!originOk(req)) throw new HttpError(403, 'Origin not allowed'); @@ -305,10 +345,27 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; c server.once('error', reject); server.listen(config.port, config.bind, resolve); }); + + // STUN on UDP. Failing to open it (port taken, no permission) only costs screen sharing its helper. + let stun: dgram.Socket | null = null; + if (config.stunPort != null) { + const socket = dgram.createSocket(net.isIPv4(config.stunBind) ? 'udp4' : 'udp6'); + socket.on('message', (msg, from) => { + const answer = stunResponse(msg, from.address, from.port); + if (answer) socket.send(answer, from.port, from.address); + }); + await new Promise(resolve => { + socket.once('error', () => { socket.close(); resolve(); }); + socket.bind(config.stunPort!, config.stunBind, () => { socket.removeAllListeners('error'); socket.on('error', () => {}); stun = socket; stunPort = socket.address().port; resolve(); }); + }); + } + return { port: (server.address() as net.AddressInfo).port, + stunPort, close: () => new Promise(resolve => { clearInterval(sweep); + (stun as dgram.Socket | null)?.close(); for (const ws of wss.clients) ws.terminate(); wss.close(); server.close(() => resolve()); diff --git a/src/lib/share.svelte.ts b/src/lib/share.svelte.ts index 7413365..c32e893 100644 --- a/src/lib/share.svelte.ts +++ b/src/lib/share.svelte.ts @@ -5,7 +5,8 @@ import { SHARE, SHARE_DATA_ID, encodeShare, ShareAssembler, type ShareType } from '../core/share-signal.ts'; import type { User } from '../core/client.ts'; import type { Session } from './session.svelte.ts'; -import { desktop } from './native.ts'; +import { desktop, isWeb } from './native.ts'; +import { proxyStun } from './web.svelte.ts'; import { store } from './settings.svelte.ts'; import { ui } from './ui.svelte.ts'; @@ -51,7 +52,10 @@ class ScreenShare { private iceServers(): RTCIceServer[] { const stun = store.settings.stunServer.trim(); - return stun ? [{ urls: /^stuns?:/.test(stun) ? stun : `stun:${stun}` }] : []; + if (stun) return [{ urls: /^stuns?:/.test(stun) ? stun : `stun:${stun}` }]; + // The browser build falls back to the proxy it is served from + const own = isWeb ? proxyStun() : null; + return own ? [{ urls: own }] : []; } private others(s: Session): number[] { diff --git a/src/lib/web.svelte.ts b/src/lib/web.svelte.ts index 810d551..737c717 100644 --- a/src/lib/web.svelte.ts +++ b/src/lib/web.svelte.ts @@ -135,14 +135,17 @@ export interface ProxyServer { host: string; port: number; label: string } class ProxyInfo { servers = $state([]); any = $state(false); + // UDP port of the proxy's STUN responder, if it runs one + stun = $state(null); loaded = $state(false); error = $state(''); async load(): Promise { try { - const c = await call<{ servers: ProxyServer[]; any: boolean }>('config'); + const c = await call<{ servers: ProxyServer[]; any: boolean; stun?: number | null }>('config'); this.servers = c.servers; this.any = c.any; + this.stun = c.stun ?? null; this.error = ''; } catch (e) { this.error = (e as Error).message; @@ -151,3 +154,9 @@ class ProxyInfo { } } export const proxyInfo = new ProxyInfo(); + +// The proxy's own STUN address: a host the user already uses, so nothing new is contacted +export function proxyStun(): string | null { + if (!proxyInfo.stun || !base) return null; + try { return `stun:${new URL(base).hostname}:${proxyInfo.stun}`; } catch { return null; } +} diff --git a/test/e2e/web-shell.cjs b/test/e2e/web-shell.cjs index 9fdf083..eb30fdd 100644 --- a/test/e2e/web-shell.cjs +++ b/test/e2e/web-shell.cjs @@ -1,5 +1,5 @@ // A bare browser window for the web E2E: no preload, so the page runs as it would in a browser. -const { app, BrowserWindow, session } = require('electron'); +const { app, BrowserWindow, desktopCapturer, session } = require('electron'); const fs = require('node:fs'); const path = require('node:path'); @@ -11,6 +11,12 @@ app.whenReady().then(() => { item.setSavePath(file); item.once('done', (_ev, state) => { if (state === 'completed') fs.writeFileSync(file + '.done', ''); }); }); + // Stands in for the browser's own "choose what to share" dialog: always the first screen + session.defaultSession.setDisplayMediaRequestHandler(async (_request, callback) => { + let sources = []; + for (let i = 0; i < 10 && !sources.length; i++) sources = await desktopCapturer.getSources({ types: ['screen'] }); + try { callback(sources.length ? { video: sources[0] } : {}); } catch { /* refused */ } + }); const win = new BrowserWindow({ width: 1280, height: 800, webPreferences: { contextIsolation: true, sandbox: true, nodeIntegration: false } }); win.loadURL(process.env.MUMH5_WEB_URL); }); diff --git a/test/e2e/web.e2e.ts b/test/e2e/web.e2e.ts index 17bbad5..5f34787 100644 --- a/test/e2e/web.e2e.ts +++ b/test/e2e/web.e2e.ts @@ -42,7 +42,7 @@ function within(p: Promise, label: string, ms = 10000): Promise { const proxyPort = 18000 + Math.floor(Math.random() * 1000); const { ELECTRON_RUN_AS_NODE, ...env } = process.env; const proxy = spawn(process.execPath, [path.join(root, 'dist-proxy/proxy.mjs')], { - env: { ...env, MUMH5_PORT: String(proxyPort), MUMH5_SERVERS: `${target}=Test Server` }, stdio: ['ignore', 'pipe', 'inherit'] + env: { ...env, MUMH5_PORT: String(proxyPort), MUMH5_SERVERS: `${target}=Test Server`, MUMH5_STUN_PORT: String(proxyPort + 1000), MUMH5_STUN_BIND: '127.0.0.1' }, stdio: ['ignore', 'pipe', 'inherit'] }); await within(new Promise((res, rej) => { proxy.stdout.on('data', d => { if (String(d).includes('listening')) res(); }); @@ -50,12 +50,14 @@ await within(new Promise((res, rej) => { }), 'proxy start'); const downloads = mkdtempSync(path.join(tmpdir(), 'mumh5-e2e-dl-')); -const app = await electron.launch({ +const browser = () => electron.launch({ executablePath: electronPath as unknown as string, args: [path.join(root, 'test/e2e/web-shell.cjs'), `--user-data-dir=${mkdtempSync(path.join(tmpdir(), 'mumh5-e2e-'))}`, '--ozone-platform=x11', '--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'], env: { ...env, MUMH5_WEB_URL: `http://127.0.0.1:${proxyPort}/`, MUMH5_DOWNLOADS: downloads } as Record }); +const app = await browser(); +let second: Awaited> | null = null; const bob = await headless('bob'); const name = `webalice${Date.now() % 100000}`; try { @@ -160,9 +162,38 @@ try { assert.equal(hash(), before, 'same certificate after a reload'); console.log('ok: identity kept across a reload'); + // Screen sharing between two browsers: the proxy's own STUN is used without any setting + assert.equal((await (await fetch(`http://127.0.0.1:${proxyPort}/api/config`)).json()).stun, proxyPort + 1000); + second = await browser(); + const page2 = await second.firstWindow(); + page2.on('console', m => { if (m.type() === 'error') console.log('[page2]', m.text()); }); + await page2.setViewportSize({ width: 1280, height: 800 }); + await page2.getByRole('button', { name: /Create a new identity/ }).click(); + await page2.getByLabel('Name', { exact: true }).fill(`webcarol${Date.now() % 100000}`); + await page2.getByRole('button', { name: 'Create', exact: true }).click(); + await page2.getByRole('button', { name: 'Skip for now' }).click(); + await page2.getByRole('button', { name: 'Done' }).click(); + await page2.getByTitle('Add a server').click(); + await page2.getByRole('button', { name: 'Save and connect' }).click(); + await page2.getByText(/Connected/).first().waitFor(); + + await page.locator('.me').getByRole('button', { name: 'Share your screen' }).click(); + await page.getByRole('button', { name: 'Continue' }).click(); + const dialog = page.getByRole('dialog', { name: 'Share your screen' }); + await dialog.getByRole('button', { name: 'Choose a screen or window...' }).click(); + await page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 60000 }); + await dialog.getByRole('button', { name: 'Start sharing' }).click(); + await page.getByText('You are sharing your screen').waitFor(); + await page2.locator('.stage .tile', { hasText: name }).click(); + await page2.getByRole('button', { name: 'Continue' }).click(); + await page2.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 30000 }); + await page.getByText('You are sharing your screen, 1 watching').waitFor(); + console.log('ok: screen sharing between two browsers'); + console.log('WEB E2E PASSED'); } finally { bob.disconnect(); await app.close().catch(() => {}); + await second?.close().catch(() => {}); proxy.kill(); } diff --git a/test/proxy.test.ts b/test/proxy.test.ts index 4ed867f..33eabef 100644 --- a/test/proxy.test.ts +++ b/test/proxy.test.ts @@ -2,8 +2,10 @@ // real Mumble server and are skipped unless MUMBLE_TEST_HOST is set (see server.test.ts). import { test } from 'node:test'; import net from 'node:net'; +import dgram from 'node:dgram'; import assert from 'node:assert/strict'; import { proxyLine } from '../electron/tls-transport.ts'; +import { stunResponse } from '../server/proxy.ts'; import { startProxy, defaults, parseServers, isPrivateAddress, type ProxyConfig } from '../server/proxy.ts'; import { WebSocket as WsClient } from 'ws'; import { WebSocketTransport } from '../src/core/ws-transport.ts'; @@ -40,7 +42,7 @@ test('refuses to start without allowed servers', async () => { test('config lists the allowed servers', async () => { await withProxy({ servers: parseServers('voice.example.org=Friends') }, async base => { - assert.deepEqual(await (await fetch(`${base}/api/config`)).json(), { servers: [{ host: 'voice.example.org', port: 64738, label: 'Friends' }], any: false }); + assert.deepEqual(await (await fetch(`${base}/api/config`)).json(), { servers: [{ host: 'voice.example.org', port: 64738, label: 'Friends' }], any: false, stun: null }); }); }); @@ -64,6 +66,47 @@ test('identity create, describe, export and import round trip', async () => { }); }); +// A binding request: type, length 0, magic cookie, 12 byte transaction id +const stunRequest = () => Uint8Array.from([0, 1, 0, 0, 0x21, 0x12, 0xa4, 0x42, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]); + +test('STUN answers carry the sender address, masked as the protocol asks', () => { + const v4 = stunResponse(stunRequest(), '203.0.113.7', 54321)!; + assert.deepEqual([...v4.subarray(0, 4)], [1, 1, 0, 12]); + assert.deepEqual([...v4.subarray(4, 20)], [...stunRequest().subarray(4, 20)]); + assert.deepEqual([...v4.subarray(20, 26)], [0, 0x20, 0, 8, 0, 1]); + assert.equal(((v4[26] << 8) | v4[27]) ^ 0x2112, 54321); + assert.deepEqual([...v4.subarray(28)].map((b, i) => b ^ [0x21, 0x12, 0xa4, 0x42][i]), [203, 0, 113, 7]); + // An IPv4 sender seen through an IPv6 socket is still IPv4 + assert.deepEqual([...stunResponse(stunRequest(), '::ffff:203.0.113.7', 54321)!], [...v4]); + const v6 = stunResponse(stunRequest(), '2001:db8::7', 1)!; + assert.equal(v6[25], 2); + const mask = stunRequest().subarray(4, 20); + assert.deepEqual([...v6.subarray(28)].map((b, i) => b ^ mask[i]), [0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7]); + // Not a binding request + assert.equal(stunResponse(new Uint8Array(20), '203.0.113.7', 1), null); + assert.equal(stunResponse(stunRequest().subarray(0, 12), '203.0.113.7', 1), null); +}); + +test('the proxy answers STUN over UDP and announces the port', async () => { + const proxy = await startProxy({ ...defaults, port: 0, origins: ['*'], servers: parseServers('voice.example.org'), stunPort: 0, stunBind: '127.0.0.1' }); + try { + assert.ok(proxy.stunPort); + assert.equal((await (await fetch(`http://127.0.0.1:${proxy.port}/api/config`)).json()).stun, proxy.stunPort); + const client = dgram.createSocket('udp4'); + const answer = await new Promise((resolve, reject) => { + client.once('message', resolve); + client.once('error', reject); + client.send(stunRequest(), proxy.stunPort!, '127.0.0.1'); + setTimeout(() => reject(new Error('no STUN answer')), 3000); + }); + assert.equal(((answer[26] << 8) | answer[27]) ^ 0x2112, client.address().port); + assert.deepEqual([...answer.subarray(28)].map((b, i) => b ^ [0x21, 0x12, 0xa4, 0x42][i]), [127, 0, 0, 1]); + client.close(); + } finally { + await proxy.close(); + } +}); + test('requests from other origins are refused', async () => { await withProxy({ servers: parseServers('voice.example.org'), origins: [] }, async base => { const cross = await post(base, 'identity/create', { name: 'x' }, { Origin: 'https://evil.example' }); From 5159d8a51847d9da765db62c2d0284bda5c1f39d Mon Sep 17 00:00:00 2001 From: Kibi Kelburton Date: Thu, 1 Oct 2026 21:41:42 +0200 Subject: [PATCH 5/5] Add a Docker image and compose file for the web proxy Co-Authored-By: Claude Opus 5.5 --- .dockerignore | 10 ++++++++++ .gitignore | 1 + Dockerfile | 18 ++++++++++++++++++ README.md | 9 +++++++++ docker-compose.yml | 16 ++++++++++++++++ 5 files changed, 54 insertions(+) create mode 100644 .dockerignore create mode 100644 Dockerfile create mode 100644 docker-compose.yml diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..86f871c --- /dev/null +++ b/.dockerignore @@ -0,0 +1,10 @@ +node_modules +dist +dist-web +dist-proxy +dist-electron +release +.git +docs +*.log +.env diff --git a/.gitignore b/.gitignore index 95533f8..f853b22 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ dist-proxy/ dist-electron/ release/ *.log +.env diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..7da8a72 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,18 @@ +# The browser version: web app and proxy in one image. +# docker compose up -d --build +FROM node:22-slim AS build +WORKDIR /src +COPY package.json package-lock.json ./ +# No install scripts: the Electron download they would trigger is only needed for the desktop app +RUN npm ci --ignore-scripts +COPY . . +RUN npm run build:web + +FROM node:22-slim +WORKDIR /app +COPY --from=build /src/dist-web ./dist-web +COPY --from=build /src/dist-proxy ./dist-proxy +USER node +ENV MUMH5_PORT=8080 +EXPOSE 8080/tcp 3478/udp +CMD ["node", "dist-proxy/proxy.mjs"] diff --git a/README.md b/README.md index d827394..2886218 100644 --- a/README.md +++ b/README.md @@ -168,6 +168,15 @@ MUMH5_SERVERS="mumble.example.com=My server" node dist-proxy/proxy.mjs Then open `http://127.0.0.1:8080`. For development, `npm run dev:web` starts the proxy and a hot-reloading page together, with any server allowed. To deploy, copy `dist-web/` and `dist-proxy/` next to each other on the server (Node 22 or newer, no `node_modules` needed) and put a reverse proxy with HTTPS in front that forwards WebSocket upgrades. Browsers only allow the microphone on HTTPS pages (or on localhost). +With Docker, the same thing is one command; settings go in a `.env` file next to `docker-compose.yml`: + +```bash +echo 'MUMH5_SERVERS=mumble.example.com=My server' > .env +docker compose up -d --build +``` + +The container uses the host's network: the proxy listens on `127.0.0.1:8080` for your reverse proxy and on UDP 3478 for STUN, and a Mumble server on the same machine is reachable as `localhost`. + | Variable | Default | Meaning | | --- | --- | --- | | `MUMH5_SERVERS` | none | Mumble servers people may connect to: `host[:port][=Label]`, comma-separated. Required unless `MUMH5_ALLOW_ANY=1` | diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..461363c --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,16 @@ +# The mumh5 web proxy. Settings come from a .env file next to this one, for example: +# MUMH5_SERVERS=mumble.example.com=My server +# MUMH5_TRUST_PROXY=1 +services: + mumh5: + build: . + restart: unless-stopped + # Host networking, so STUN sees each visitor's real address (through Docker's port + # forwarding it would often see Docker's own) and a Mumble server on this machine is + # reachable as localhost. The proxy listens on 127.0.0.1:8080 for nginx and on UDP 3478. + network_mode: host + env_file: + - path: .env + required: false + environment: + MUMH5_BIND: ${MUMH5_BIND:-127.0.0.1}