diff --git a/CHANGELOG.md b/CHANGELOG.md index 543a0a8..0567791 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ All notable changes to mumh5. Versions follow the `version` in `package.json`. ### Added +- Screen sharing between mumh5 users in a channel (picture only so far): a toolbar button starts it, others click the indicator next to your name to watch. Streams go directly between clients; a STUN server can be set in Settings, Voice. - Browser version: `npm run build:web` builds the web app and a small self-hosted proxy that bridges browsers to Mumble servers on an allowlist. Identities are kept in the browser, voice goes through the TCP tunnel. See "Browser version" in the README. - Link previews in chat: title, description and image for web links (up to two per message). By default they are fetched by your f0ckm upload host, so the linked sites never see your IP address. Can be switched to "fetched by this computer" or off in Settings, Chat and files. - Voice tiles on small windows: when the window is too narrow for the member list, the people in your voice channel appear as tiles above the chat, light up while they talk, and keep mute and deafen at hand. The tiles can be collapsed. diff --git a/CLAUDE.md b/CLAUDE.md index 9759383..04efade 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,6 +26,7 @@ npm run build:web # browser build to dist-web/, proxy bundle to dist-proxy/ npm run dev:web # browser build with hot reload: starts the proxy (any server allowed) and Vite npm run proxy # run the proxy from source (MUMH5_SERVERS=host:port required) npm run test:e2e:web # drives the browser build through the proxy (needs npm run build:web first) +npm run test:e2e:share # screen sharing between two instances of the built app npm run proto # regenerate src/core/mumble-pb.js and mumble-udp-pb.js from proto/ ``` @@ -46,6 +47,7 @@ A reused test server keeps registrations and channels from earlier runs; tests m - `src/core/` (browser-safe TypeScript, also runs in Node for tests): framing and codec (`proto.ts`), the Mumble client state machine (`client.ts`), voice packet formats (`voice-packet.ts`). No DOM, no Electron, no Node imports here. - `src/lib/native.ts`: `desktop` is the Electron preload API or null; `native` is what both platforms provide (identities, certificates), backed by `web.svelte.ts` in the browser build (`isWeb`, vite `--mode web`). Desktop-only features check `desktop`. - `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets). `html.ts` sanitizes incoming HTML and serializes outgoing rich text. +- Screen sharing: `src/core/share-signal.ts` (signals as Mumble plugin data, id `mumh5.share`, compressed and chunked) and `src/lib/share.svelte.ts` (one WebRTC connection per viewer, candidates inside the description, no trickle). The desktop source picker is `share:sources` / `share:pick` in `electron/main.ts`. - `src/ui/`: Svelte components. `App.svelte` owns layout and global dialogs (`ui.svelte.ts` store). Voice runs on one server at a time (where you last joined a channel); background servers are auto self-deafened and restored when voice returns. @@ -62,6 +64,7 @@ Voice runs on one server at a time (where you last joined a channel); background - The CSP allows media only from `self`, `blob:` and http(s). `data:` audio is blocked silently: play user files through object URLs. Large user files go to IndexedDB (`src/lib/blobstore.ts`), not localStorage. - `window.prompt` does not exist in Electron; use `ui.prompt`. +- `desktopCapturer.getSources` can return an empty list on its first calls under X11; main retries. On Wayland the call itself opens the system picker, so it is never repeated. - A drop handler must read derived state before clearing the drag item it derives from. ## Svelte reactivity pitfalls (all hit before) @@ -76,6 +79,7 @@ Voice runs on one server at a time (where you last joined a channel); background - Murmur silently drops own UserState, TextMessage, ChannelState, ACL and Version past a leaky bucket (burst 5, 1/s). The client paces these; don't bypass `send()`. - 1.5 servers use the protobuf UDP voice format (type byte 0 + MumbleUDP.Audio) with clients announcing 1.5; older ones the legacy format. The UDPTunnel TCP body is the raw voice packet, not a protobuf message. Sequence numbers count 10 ms frames. - Long comments and descriptions arrive as a hash only; a new hash invalidates the old text; fetch with RequestBlob. +- Murmur drops PluginDataTransmission over 1000 bytes and rate limits it; keep packets at 900 bytes and few. - Renaming while connected is not possible; mumh5 reconnects with the new name. - Murmur never sends SuperUser (user id 0) PermissionQuery answers; treat SuperUser as allowed everything. - In ACLs, Write overrides denies. Grant test rights to one user (`$` group), not `@all`. diff --git a/README.md b/README.md index 0124713..9800877 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,14 @@ mumh5 keeps the foundation and replaces the experience. ![Profile with a formatted description](docs/screenshots/profile.png) +### Screen sharing + +- Share a screen or a window with the mumh5 users in your channel; they see an indicator next to your name and click it to watch +- No server setup and no extra account: the setup messages travel through the Mumble server, the picture goes directly between the two clients (WebRTC) +- Picture only for now, up to 8 viewers. Regular Mumble clients do not see streams +- Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Behind most home routers a STUN server (Settings, Voice) is needed; none is contacted unless you enter one +- Tested between two desktop instances on one machine; connections across the internet and the browser build are untested + ### Chat - Messages grouped by author, direct messages, unread badges - File sharing: drag and drop, paste, or the attach button, with upload progress @@ -254,7 +262,7 @@ Without an upload host, mumh5 still sends images, scaled to fit the server's lim - System-wide push to talk - Whisper and shout - Rich chat between mumh5 users: replies, reactions, edits, typing indicators -- Screen sharing between mumh5 users in a channel, with sound (one application or the whole system, also on Linux through PipeWire). Anyone can start a stream; no extra key, your Mumble certificate is your identity. Streams go directly between clients with WebRTC, set up through the Mumble server, with an optional self-hosted relay for many viewers +- Screen sharing: sound (one application or the whole system, also on Linux through PipeWire), quality settings, and an optional self-hosted relay for many viewers - Release builds for all platforms --- diff --git a/electron/main.ts b/electron/main.ts index f3aba65..c33439d 100644 --- a/electron/main.ts +++ b/electron/main.ts @@ -1,4 +1,4 @@ -import { app, BrowserWindow, dialog, ipcMain, net, shell, session, systemPreferences, type WebContents } from 'electron'; +import { app, BrowserWindow, desktopCapturer, dialog, ipcMain, net, shell, session, systemPreferences, type WebContents } from 'electron'; import { promises as fs } from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -12,6 +12,11 @@ import { udpChannel } from './udp-voice.ts'; const devUrl = process.env.VITE_DEV_SERVER_URL; +// Screen sharing connects mumh5 users directly. Chromium would hide this computer's addresses +// behind names that only resolve on the local network, so two people on IPv6 could not reach +// each other without a helper server. Only this app's own page runs here. +app.commandLine.appendSwitch('disable-features', 'WebRtcHideLocalIpsWithMdns'); + let identityStore: IdentityStore | null = null; const identities = () => (identityStore ??= new IdentityStore(app.getPath('userData'))); @@ -120,6 +125,30 @@ ipcMain.handle('publist:ping', (_e, host: string, port: number) => pingServer(St ipcMain.handle('preview:fetch', (_e, url: string) => fetchLinkPreview(String(url).slice(0, 2048))); +// ─── Screen sharing ─────────────────────────────────────────────────────────── +// The renderer lists the sources, the user picks one, and the next getDisplayMedia call gets it. +// On Wayland, listing already opens the system's picker and returns the one chosen source; +// `chosen` tells the renderer not to ask a second time. +let screenSources: Electron.DesktopCapturerSource[] = []; +let pickedSource: string | null = null; +ipcMain.handle('share:sources', async () => { + const wayland = process.platform === 'linux' && app.commandLine.getSwitchValue('ozone-platform') !== 'x11' && + (process.env.XDG_SESSION_TYPE === 'wayland' || !!process.env.WAYLAND_DISPLAY); + const list = () => desktopCapturer.getSources({ types: ['screen', 'window'], thumbnailSize: { width: 320, height: 180 } }); + screenSources = await list(); + // The X11 capturer can come back empty on its first calls. Not on Wayland, where empty + // means the user cancelled the system's picker and asking again would reopen it. + for (let i = 0; i < 8 && !screenSources.length && !wayland; i++) { + await new Promise(r => setTimeout(r, 250)); + screenSources = await list(); + } + return { + chosen: wayland && screenSources.length === 1, + sources: screenSources.map(s => ({ id: s.id, name: s.name, thumbnail: s.thumbnail.isEmpty() ? '' : s.thumbnail.toDataURL() })) + }; +}); +ipcMain.on('share:pick', (_e, id: string) => { pickedSource = String(id); }); + ipcMain.on('tray:update', (_e, state: tray.TrayState) => tray.update(state)); ipcMain.handle('platform:info', () => ({ os: process.platform, osVersion: os.release() })); @@ -195,6 +224,13 @@ app.whenReady().then(async () => { cb({ requestHeaders: details.requestHeaders }); }); + session.defaultSession.setDisplayMediaRequestHandler((_request, callback) => { + const source = screenSources.find(s => s.id === pickedSource); + pickedSource = null; + // Without a picked source the request is refused + try { callback(source ? { video: source } : {}); } catch { /* refused */ } + }); + // Microphone, camera and screen capture for voice and video; nothing else session.defaultSession.setPermissionRequestHandler((_wc, permission, cb) => { cb(['media', 'display-capture', 'clipboard-sanitized-write', 'notifications'].includes(permission)); diff --git a/electron/preload.ts b/electron/preload.ts index 704d66c..e8c2b76 100644 --- a/electron/preload.ts +++ b/electron/preload.ts @@ -21,6 +21,8 @@ contextBridge.exposeInMainWorld('mumh5Native', { onContextMenu: (fn: (params: unknown) => void) => { ipcRenderer.on('context-menu', (_e, p) => fn(p)); }, editAction: (action: string, arg?: unknown) => ipcRenderer.send('edit:action', action, arg), onTrayAction: (fn: (action: string) => void) => { ipcRenderer.on('tray:action', (_e, a: string) => fn(a)); }, + screenSources: () => ipcRenderer.invoke('share:sources'), + pickScreenSource: (id: string) => ipcRenderer.send('share:pick', id), describeCerts: (ders: Uint8Array[]) => ipcRenderer.invoke('certs:describe', ders), identities: { list: () => ipcRenderer.invoke('identities:list'), diff --git a/package.json b/package.json index 0504d72..78d7e2b 100644 --- a/package.json +++ b/package.json @@ -20,6 +20,7 @@ "dist:mac": "npm run build && electron-builder --mac", "test:e2e": "node test/e2e/app.e2e.ts", "test:e2e:web": "node test/e2e/web.e2e.ts", + "test:e2e:share": "node test/e2e/share.e2e.ts", "proto": "pbjs -t static-module -w es6 --keep-case --no-delimited --no-service --no-comments --force-number proto/Mumble.proto -o src/core/mumble-pb.js && pbjs -t static-module -w es6 --keep-case --no-delimited --no-service --no-comments --force-number proto/MumbleUDP.proto -o src/core/mumble-udp-pb.js", "screenshots": "node test/e2e/screenshots.ts" }, diff --git a/src/App.svelte b/src/App.svelte index 82cc44f..e39257b 100644 --- a/src/App.svelte +++ b/src/App.svelte @@ -15,6 +15,7 @@ import ChannelDialog from './ui/ChannelDialog.svelte'; import PublicServers from './ui/PublicServers.svelte'; import ServerInfoDialog from './ui/ServerInfoDialog.svelte'; + import SharePicker from './ui/SharePicker.svelte'; import ResizeHandle from './ui/ResizeHandle.svelte'; import { identities } from './lib/identities.svelte.ts'; import { session } from './lib/session.svelte.ts'; @@ -129,7 +130,8 @@ {#if ui.channelDialog && session.status === 'connected'} {#key ui.channelDialog} (ui.channelDialog = null)} />{/key} {/if} -{#if ui.prompt} (ui.prompt = null)} />{/if} + +{#if ui.prompt} { const p = ui.prompt; ui.prompt = null; p?.oncancel?.(); }} />{/if} diff --git a/src/ui/Chat.svelte b/src/ui/Chat.svelte index 4fc7532..ff9590a 100644 --- a/src/ui/Chat.svelte +++ b/src/ui/Chat.svelte @@ -3,6 +3,7 @@ import ChatLog from './ChatLog.svelte'; import Composer from './Composer.svelte'; import VoiceStage from './VoiceStage.svelte'; + import ShareView from './ShareView.svelte'; import { ui } from '../lib/ui.svelte.ts'; import { session } from '../lib/session.svelte.ts'; @@ -28,6 +29,7 @@ {#if showStage && session.status === 'connected'}{/if} + {#if session.status === 'connected'}{/if} {#if session.status === 'idle' && session.disconnectInfo} {@const d = session.disconnectInfo} diff --git a/src/ui/SettingsDialog.svelte b/src/ui/SettingsDialog.svelte index 1d30326..9c495f9 100644 --- a/src/ui/SettingsDialog.svelte +++ b/src/ui/SettingsDialog.svelte @@ -59,6 +59,10 @@ {#if tab === 'voice'} +

Screen sharing

+ + store.saveSettings()} placeholder="stun.example.com:3478" autocomplete="off" /> +

Screen sharing connects you directly to the people watching. Behind most home routers that needs a STUN server to find a route; it learns your IP address but never sees the stream. Empty means no outside server is contacted, which works on the same network and often over IPv6.

{:else if tab === 'sounds'} {:else if tab === 'appearance'} @@ -119,6 +123,7 @@ .tabs button { padding: 8px 14px; color: var(--text-dim); font-weight: 600; white-space: nowrap; } .tabs button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); } h3 { margin: 22px 0 0; font-size: 14px; } + .opt { text-transform: none; font-weight: 400; letter-spacing: 0; color: var(--text-faint); } .help { margin: 10px 0 0; font-size: 13px; color: var(--text-dim); } .row { display: flex; align-items: flex-end; gap: 8px; } .row > div { flex: 1; } diff --git a/src/ui/SharePicker.svelte b/src/ui/SharePicker.svelte new file mode 100644 index 0000000..7a39ff1 --- /dev/null +++ b/src/ui/SharePicker.svelte @@ -0,0 +1,35 @@ + + +{#if picker} + picker.choose(null)} width={720}> +

Pick a screen or a window. mumh5 users in your channel can then watch it.

+
    + {#each picker.sources as s (s.id)} +
  • + +
  • + {/each} +
+ {#snippet footer()} + + {/snippet} +
+{/if} + + diff --git a/src/ui/ShareView.svelte b/src/ui/ShareView.svelte new file mode 100644 index 0000000..e9dc267 --- /dev/null +++ b/src/ui/ShareView.svelte @@ -0,0 +1,60 @@ + + +{#if mine} +
+ + You are sharing your screen{share.viewers ? `, ${share.viewers} watching` : ''} + +
+{/if} +{#if share.error}{/if} + +{#if watching} +
+
+ + {sharer ? store.displayName(sharer) : 'Screen share'} + + +
+
+ + + {#if watching.state === 'connecting'}

Connecting...

+ {:else if watching.state === 'failed'}

Could not connect. You and the other person may both be behind routers that block direct connections; a STUN server in Settings, Voice can help.

{/if} +
+
+{/if} + + diff --git a/src/ui/Sidebar.svelte b/src/ui/Sidebar.svelte index 6ccfb94..dbef956 100644 --- a/src/ui/Sidebar.svelte +++ b/src/ui/Sidebar.svelte @@ -9,6 +9,7 @@ import { PERM } from '../core/client.ts'; import ResizeHandle from './ResizeHandle.svelte'; import { voice } from '../lib/audio/voice.svelte.ts'; + import { share } from '../lib/share.svelte.ts'; // resizable: false in the phone drawer, which keeps a fixed width // handleEdge: which side the resize handle sits on (left when the list is on the right, classic layout) @@ -50,6 +51,14 @@ ui.panelOpen = true; } + // Sharing runs on one server at a time; the button stops it from anywhere + const sharingHere = $derived(!!share.stream); + const canShare = $derived(session.status === 'connected' && share.supported(sessions.active)); + function toggleShare() { + if (share.stream) share.stop(); + else share.start(sessions.active); + } + // One button for the whole tree: open everything, then close everything let allOpen = $state(false); function toggleAll() { @@ -159,6 +168,10 @@ onclick={() => session.setSelfDeaf(!deafened)} oncontextmenu={e => openQuick(e, 'output')}> +