Compare commits
37
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
06a185d778 | ||
|
|
c990593f9f | ||
|
|
e41a2cfd70 | ||
|
|
fdf6c12afa | ||
|
|
5d43b8e4ae | ||
|
|
6368e63b0e | ||
|
|
2700a4ec3d | ||
|
|
1e4bda4e74 | ||
|
|
817f822771 | ||
|
|
819db06bc6 | ||
|
|
08aa5268d0 | ||
|
|
98e44d7082 | ||
|
|
0a290de229 | ||
|
|
2510633538 | ||
|
|
e7b317313a | ||
|
|
f9e7042a73 | ||
|
|
7d1ad5423f | ||
|
|
b80eeafcc2 | ||
|
|
9832ba2749 | ||
|
|
950a1afad7 | ||
|
|
b5dda4e1c2 | ||
|
|
e0e900625d | ||
|
|
e59c6804cb | ||
|
|
5cc379417b | ||
|
|
68f68d1d76 | ||
|
|
a402d1c035 | ||
|
|
63d77c2bce | ||
|
|
9880ef840f | ||
|
|
71b90ef80a | ||
|
|
23833b1c8b | ||
|
|
5a7f243c1d | ||
|
|
e649062d67 | ||
|
|
5159d8a518 | ||
|
|
826542ed8a | ||
|
|
13b7013004 | ||
|
|
3b23f69a5f | ||
|
|
0cce3c97c2 |
@@ -0,0 +1,10 @@
|
||||
node_modules
|
||||
dist
|
||||
dist-web
|
||||
dist-proxy
|
||||
dist-electron
|
||||
release
|
||||
.git
|
||||
docs
|
||||
*.log
|
||||
.env
|
||||
@@ -5,3 +5,4 @@ dist-proxy/
|
||||
dist-electron/
|
||||
release/
|
||||
*.log
|
||||
.env
|
||||
|
||||
@@ -6,6 +6,11 @@ All notable changes to mumh5. Versions follow the `version` in `package.json`.
|
||||
|
||||
### Added
|
||||
|
||||
- Voice in browsers without an audio codec of their own (Firefox on Android): a built-in Opus codec takes over there.
|
||||
- Cameras of people in your channel show by themselves, like in a video call; each one can be hidden from its right-click menu, and all of them switched off with one button.
|
||||
- Camera sharing, as a second source in the screen sharing dialog, and camera rounds: several people's cameras at once in the voice tiles.
|
||||
- A relay for streams in the web proxy, used automatically by the browser version and, with its address entered in Settings, by the desktop app, for networks and browsers that allow no direct connection.
|
||||
- Screen sharing between mumh5 users in a channel: a dialog to choose the screen or window (with preview), the sound and the quality; sound from one program or the whole system on Linux through PipeWire. Viewers watch in the voice tiles, enlarge a stream like in a meeting and set its volume. Streams go directly between clients; a STUN server can be set in Settings, Voice.
|
||||
- Browser version: `npm run build:web` builds the web app and a small self-hosted proxy that bridges browsers to Mumble servers on an allowlist. Identities are kept in the browser, voice goes through the TCP tunnel. See "Browser version" in the README.
|
||||
- Link previews in chat: title, description and image for web links (up to two per message). By default they are fetched by your f0ckm upload host, so the linked sites never see your IP address. Can be switched to "fetched by this computer" or off in Settings, Chat and files.
|
||||
- Voice tiles on small windows: when the window is too narrow for the member list, the people in your voice channel appear as tiles above the chat, light up while they talk, and keep mute and deafen at hand. The tiles can be collapsed.
|
||||
|
||||
@@ -26,6 +26,7 @@ npm run build:web # browser build to dist-web/, proxy bundle to dist-proxy/
|
||||
npm run dev:web # browser build with hot reload: starts the proxy (any server allowed) and Vite
|
||||
npm run proxy # run the proxy from source (MUMH5_SERVERS=host:port required)
|
||||
npm run test:e2e:web # drives the browser build through the proxy (needs npm run build:web first)
|
||||
npm run test:e2e:share # screen sharing between two instances of the built app
|
||||
npm run proto # regenerate src/core/mumble-pb.js and mumble-udp-pb.js from proto/
|
||||
```
|
||||
|
||||
@@ -42,10 +43,11 @@ A reused test server keeps registrations and channels from earlier runs; tests m
|
||||
## Architecture
|
||||
|
||||
- `electron/` (Node, main process): window, TLS sockets to Mumble servers (`tls-transport.ts`), encrypted UDP voice (`udp-voice.ts`, `ocb2.ts`, tested against Mumble's OCB2 vectors), identities and PKCS#12 (`identity.ts`, `identity-store.ts`), certificate parsing (`certs.ts`), tray (`tray.ts`). The renderer only gets the narrow `window.mumh5Native` API from `preload.ts` (context isolation, sandbox).
|
||||
- `server/` (Node): the web proxy. `proxy.ts` serves `dist-web`, bridges WebSocket connections to Mumble over TLS (reusing `electron/tls-transport.ts`) and has stateless identity endpoints. It stores nothing; the browser keeps identities in localStorage and sends one with each connect.
|
||||
- `server/` (Node): the web proxy. `proxy.ts` serves `dist-web`, bridges WebSocket connections to Mumble over TLS (reusing `electron/tls-transport.ts`) and has stateless identity endpoints. It stores nothing; the browser keeps identities in localStorage and sends one with each connect. `turn.ts` is its STUN and TURN server for screen sharing between browser users (UDP and TCP on one port, credentials from `GET /api/relay`, which any origin may read so the desktop app can use a proxy as its relay; secret made up at start). `localStorage mumh5.iceDebug = relay` or `relay-tcp` limits a client to relayed routes, which is how the E2E exercises the relay.
|
||||
- `src/core/` (browser-safe TypeScript, also runs in Node for tests): framing and codec (`proto.ts`), the Mumble client state machine (`client.ts`), voice packet formats (`voice-packet.ts`). No DOM, no Electron, no Node imports here.
|
||||
- `src/lib/native.ts`: `desktop` is the Electron preload API or null; `native` is what both platforms provide (identities, certificates), backed by `web.svelte.ts` in the browser build (`isWeb`, vite `--mode web`). Desktop-only features check `desktop`.
|
||||
- `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets). `html.ts` sanitizes incoming HTML and serializes outgoing rich text.
|
||||
- `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets); where a browser has no WebCodecs audio it uses libopus in WebAssembly (`audio/opus-wasm.ts`, `localStorage mumh5.forceWasmOpus = 1` forces it). `html.ts` sanitizes incoming HTML and serializes outgoing rich text.
|
||||
- Screen sharing: `src/core/share-signal.ts` (signals as Mumble plugin data, id `mumh5.share`, compressed and chunked) and `src/lib/share.svelte.ts` (one WebRTC connection per viewer and stream, candidates inside the description, no trickle; a viewer holds one screen and any number of cameras in `share.watches`). The start dialog is `ShareDialog.svelte`, the tiles and the large view are `VoiceStage.svelte`. Desktop source listing is `share:sources` / `share:pick` in `electron/main.ts`; Linux sound is `electron/pipewire.ts` (a virtual microphone fed with pw-link, never with this app's own playback).
|
||||
- `src/ui/`: Svelte components. `App.svelte` owns layout and global dialogs (`ui.svelte.ts` store).
|
||||
|
||||
Voice runs on one server at a time (where you last joined a channel); background servers are auto self-deafened and restored when voice returns.
|
||||
@@ -53,7 +55,7 @@ Voice runs on one server at a time (where you last joined a channel); background
|
||||
## Conventions
|
||||
|
||||
- Svelte 5 runes only. Comment density: short comments that explain why, not what.
|
||||
- The protobuf codec is generated static code (no eval) because the CSP forbids `unsafe-eval`. Field names keep snake_case from the .proto.
|
||||
- The protobuf codec is generated static code (no eval) because the CSP forbids `unsafe-eval`. `wasm-unsafe-eval` is allowed, for the built-in Opus codec only. Field names keep snake_case from the .proto.
|
||||
- TypeScript files imported by Node tests use `.ts` extensions and erasable syntax only (no parameter properties, no enums).
|
||||
- New features get an E2E step in `test/e2e/app.e2e.ts`; protocol logic gets unit or server tests.
|
||||
- Privacy rule: opening a chat must never contact hosts the user did not choose. Remote media only from the upload host and user-listed hosts; YouTube is click-to-play (`youtube-nocookie.com`).
|
||||
@@ -62,6 +64,7 @@ Voice runs on one server at a time (where you last joined a channel); background
|
||||
|
||||
- The CSP allows media only from `self`, `blob:` and http(s). `data:` audio is blocked silently: play user files through object URLs. Large user files go to IndexedDB (`src/lib/blobstore.ts`), not localStorage.
|
||||
- `window.prompt` does not exist in Electron; use `ui.prompt`.
|
||||
- `desktopCapturer.getSources` can return an empty list on its first calls under X11 and takes seconds on the bare Xvfb test display; main retries and the dialog has Refresh. With `--use-fake-device-for-media-stream` the captured picture is Chromium's test pattern, not the screen. On Wayland the call itself opens the system picker, so it is never repeated.
|
||||
- A drop handler must read derived state before clearing the drag item it derives from.
|
||||
|
||||
## Svelte reactivity pitfalls (all hit before)
|
||||
@@ -76,6 +79,7 @@ Voice runs on one server at a time (where you last joined a channel); background
|
||||
- Murmur silently drops own UserState, TextMessage, ChannelState, ACL and Version past a leaky bucket (burst 5, 1/s). The client paces these; don't bypass `send()`.
|
||||
- 1.5 servers use the protobuf UDP voice format (type byte 0 + MumbleUDP.Audio) with clients announcing 1.5; older ones the legacy format. The UDPTunnel TCP body is the raw voice packet, not a protobuf message. Sequence numbers count 10 ms frames.
|
||||
- Long comments and descriptions arrive as a hash only; a new hash invalidates the old text; fetch with RequestBlob.
|
||||
- Murmur drops PluginDataTransmission over 1000 bytes and rate limits it; keep packets at 900 bytes and few.
|
||||
- Renaming while connected is not possible; mumh5 reconnects with the new name.
|
||||
- Murmur never sends SuperUser (user id 0) PermissionQuery answers; treat SuperUser as allowed everything.
|
||||
- In ACLs, Write overrides denies. Grant test rights to one user (`$<certhash>` group), not `@all`.
|
||||
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
# The browser version: web app and proxy in one image.
|
||||
# docker compose up -d --build
|
||||
FROM node:22-slim AS build
|
||||
WORKDIR /src
|
||||
COPY package.json package-lock.json ./
|
||||
# No install scripts: the Electron download they would trigger is only needed for the desktop app
|
||||
RUN npm ci --ignore-scripts
|
||||
COPY . .
|
||||
RUN npm run build:web
|
||||
|
||||
FROM node:22-slim
|
||||
WORKDIR /app
|
||||
COPY --from=build /src/dist-web ./dist-web
|
||||
COPY --from=build /src/dist-proxy ./dist-proxy
|
||||
USER node
|
||||
ENV MUMH5_PORT=8080
|
||||
EXPOSE 8080/tcp 3478/udp 3478/tcp 49160-49659/udp
|
||||
CMD ["node", "dist-proxy/proxy.mjs"]
|
||||
@@ -76,6 +76,16 @@ mumh5 keeps the foundation and replaces the experience.
|
||||
|
||||

|
||||
|
||||
### Screen sharing
|
||||
|
||||
- Camera rounds: everyone who turns their camera on appears in the tiles at once, like a video meeting; one click joins or leaves the round, and a click on a tile enlarges that camera
|
||||
- Share a screen, a window or a camera with the mumh5 users in your channel. A dialog lets you choose what to show, with a live preview, which sound goes with it and the quality, before anything is sent
|
||||
- Sound: on Linux one program or everything except mumh5 itself (through PipeWire, so viewers do not hear the voice chat twice); on Windows the whole system; in a browser what the browser offers
|
||||
- People in the channel see an indicator next to your name, on your tile and in your profile, and click to watch. The tiles above the chat fill a stage you can drag taller or shorter. A stream opens large, with everyone as a strip of tiles below, like a meeting, and can move to a window of its own; in the stacked layout it is just the picture, as wide as the chat. Sounds announce streams and viewers. Viewers set the stream's volume
|
||||
- No server setup and no extra account: the setup messages travel through the Mumble server, the stream goes directly between the two clients (WebRTC), up to 8 viewers. Regular Mumble clients do not see streams
|
||||
- Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Across the internet both sides need a STUN server. The browser version uses the one built into its proxy, and the proxy's relay when no direct connection is possible (mobile networks, Vanadium); in the desktop app you choose a STUN server in Settings, Voice (Google, Cloudflare, or any address), and can enter the address of a mumh5 web version as a relay, which then takes over by itself when no direct connection comes up. Nothing is contacted unless you set it
|
||||
- Tested between two desktop instances on one machine with a test picture. Sound capture, real screens, connections across the internet, Windows and the browser build are untested
|
||||
|
||||
### Chat
|
||||
- Messages grouped by author, direct messages, unread badges
|
||||
- File sharing: drag and drop, paste, or the attach button, with upload progress
|
||||
@@ -159,6 +169,17 @@ MUMH5_SERVERS="mumble.example.com=My server" node dist-proxy/proxy.mjs
|
||||
|
||||
Then open `http://127.0.0.1:8080`. For development, `npm run dev:web` starts the proxy and a hot-reloading page together, with any server allowed. To deploy, copy `dist-web/` and `dist-proxy/` next to each other on the server (Node 22 or newer, no `node_modules` needed) and put a reverse proxy with HTTPS in front that forwards WebSocket upgrades. Browsers only allow the microphone on HTTPS pages (or on localhost).
|
||||
|
||||
With Docker, the same thing is one command; settings go in a `.env` file next to `docker-compose.yml`:
|
||||
|
||||
```bash
|
||||
echo 'MUMH5_SERVERS=mumble.example.com=My server' > .env
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
The container uses the host's network: the proxy listens on `127.0.0.1:8080` for your reverse proxy, and a Mumble server on the same machine is reachable as `localhost`.
|
||||
|
||||
For screen sharing, open these in the firewall (and forward them on a router in front of the server): port 3478 for UDP and TCP, and UDP 49160-49659. They carry STUN and the relay, which the browser version uses without any setting. Relayed streams pass through your server and use its bandwidth, a few Mbit/s per viewer; anyone who knows the site's address can get credentials for it (the desktop app asks from outside the site), limited by connection quotas per address; there is no bandwidth cap yet.
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `MUMH5_SERVERS` | none | Mumble servers people may connect to: `host[:port][=Label]`, comma-separated. Required unless `MUMH5_ALLOW_ANY=1` |
|
||||
@@ -167,6 +188,13 @@ Then open `http://127.0.0.1:8080`. For development, `npm run dev:web` starts the
|
||||
| `MUMH5_ORIGINS` | same host | Origins allowed to use the API, comma-separated, when the page is hosted elsewhere |
|
||||
| `MUMH5_TRUST_PROXY` | off | Take client addresses from `X-Forwarded-For` (set this behind a reverse proxy) |
|
||||
| `MUMH5_SEND_PROXY` | off | Announce each visitor's address to the server with the PROXY protocol (see below). Breaks connections to a plain Mumble server |
|
||||
| `MUMH5_STUN_PORT`, `MUMH5_STUN_BIND` | `3478`, all addresses | Port for screen sharing between browser users: STUN over UDP, and the relay over UDP and TCP. Browsers reach it directly, not through nginx. `0` turns both off |
|
||||
| `MUMH5_TURN` | on | The relay (TURN) for people who cannot connect directly: mobile networks, strict company networks, browsers that forbid direct UDP such as Vanadium. `0` leaves only STUN |
|
||||
| `MUMH5_TURN_PORTS` | `49160-49659` | UDP ports the relayed streams use, one per relayed route |
|
||||
| `MUMH5_TURN_URLS` | none | Extra relay addresses handed to clients, comma-separated, for example `turns:turn.example.com:443?transport=tcp` when a TLS front forwards to port 3478. For networks that only let port 443 out |
|
||||
| `MUMH5_DEBUG` | off | Log relay events (credentials handed out, routes granted or refused) |
|
||||
| `MUMH5_TURN_MAX`, `MUMH5_TURN_PER_ADDRESS` | `500`, `64` | Relayed routes in total and per client address. A camera round uses several per person |
|
||||
| `MUMH5_TURN_IP` | found automatically | The server's public address, announced for relayed streams. Set it when the server sits behind a 1:1 NAT, as on many cloud hosts |
|
||||
| `MUMH5_STATIC` | `../dist-web` | Folder with the web build |
|
||||
| `MUMH5_MAX_CONNECTIONS`, `MUMH5_MAX_PER_ADDRESS` | `200`, `8` | Connection limits, in total and per client address |
|
||||
|
||||
@@ -220,7 +248,7 @@ What is different from the desktop app:
|
||||
- **Voice goes over TCP.** Browsers have no UDP sockets, so voice uses Mumble's TCP tunnel through the proxy. It works, with more delay on lossy connections.
|
||||
- **Clearing the browser's data removes your identity.** Download a backup; the same `.p12` file works in the desktop app and in Mumble.
|
||||
- **Not available:** the public server browser, link previews fetched by your own computer, the tray icon, importing the desktop Mumble certificate automatically.
|
||||
- Voice needs a browser with WebCodecs audio. It is tested in Chromium only.
|
||||
- Voice uses the browser's own audio codec where there is one, and a built-in one otherwise (Firefox on Android). Both are tested in Chromium only.
|
||||
|
||||
## File sharing with f0ckm
|
||||
|
||||
@@ -254,7 +282,7 @@ Without an upload host, mumh5 still sends images, scaled to fit the server's lim
|
||||
- System-wide push to talk
|
||||
- Whisper and shout
|
||||
- Rich chat between mumh5 users: replies, reactions, edits, typing indicators
|
||||
- Screen sharing between mumh5 users in a channel, with sound (one application or the whole system, also on Linux through PipeWire). Anyone can start a stream; no extra key, your Mumble certificate is your identity. Streams go directly between clients with WebRTC, set up through the Mumble server, with an optional self-hosted relay for many viewers
|
||||
- Screen sharing: a media server for large rounds (today every sender uploads once per viewer, up to 8)
|
||||
- Release builds for all platforms
|
||||
|
||||
---
|
||||
|
||||
@@ -41,3 +41,11 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
Runtime and build dependencies (Electron, Svelte, protobufjs, node-forge, DOMPurify and
|
||||
others) are distributed under their own licenses, found in each package under `node_modules/`.
|
||||
|
||||
## @evan/opus and libopus
|
||||
|
||||
The Opus codec compiled to WebAssembly (`opus.wasm`), used for voice in browsers that have no
|
||||
audio codec of their own. The package is MIT licensed, Copyright (c) 2021-present evanwashere.
|
||||
libopus is under the three-clause BSD license, Copyright 2001-2011 Xiph.Org, Skype Limited,
|
||||
Octasic, Jean-Marc Valin, Timothy B. Terriberry, CSIRO, Gregory Maxwell, Mark Borgerding,
|
||||
Erik de Castro Lopo.
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# The mumh5 web proxy. Settings come from a .env file next to this one, for example:
|
||||
# MUMH5_SERVERS=mumble.example.com=My server
|
||||
# MUMH5_TRUST_PROXY=1
|
||||
services:
|
||||
mumh5:
|
||||
build: .
|
||||
restart: unless-stopped
|
||||
# Host networking, so STUN sees each visitor's real address (through Docker's port
|
||||
# forwarding it would often see Docker's own) and a Mumble server on this machine is
|
||||
# reachable as localhost. The proxy listens on 127.0.0.1:8080 for nginx, on port 3478 (UDP and
|
||||
# TCP) for STUN and the relay, and relays streams on UDP 49160-49659.
|
||||
network_mode: host
|
||||
# Passed on from .env (or the shell); empty means the proxy's default. Written out one by
|
||||
# one because older docker-compose versions cannot mark an env_file as optional.
|
||||
environment:
|
||||
MUMH5_SERVERS: ${MUMH5_SERVERS:-}
|
||||
MUMH5_ALLOW_ANY: ${MUMH5_ALLOW_ANY:-}
|
||||
MUMH5_ALLOW_PRIVATE: ${MUMH5_ALLOW_PRIVATE:-}
|
||||
MUMH5_PORT: ${MUMH5_PORT:-8080}
|
||||
MUMH5_BIND: ${MUMH5_BIND:-127.0.0.1}
|
||||
MUMH5_ORIGINS: ${MUMH5_ORIGINS:-}
|
||||
MUMH5_TRUST_PROXY: ${MUMH5_TRUST_PROXY:-}
|
||||
MUMH5_SEND_PROXY: ${MUMH5_SEND_PROXY:-}
|
||||
MUMH5_STUN_PORT: ${MUMH5_STUN_PORT:-3478}
|
||||
MUMH5_STUN_BIND: ${MUMH5_STUN_BIND:-}
|
||||
MUMH5_TURN: ${MUMH5_TURN:-}
|
||||
MUMH5_TURN_IP: ${MUMH5_TURN_IP:-}
|
||||
MUMH5_TURN_PORTS: ${MUMH5_TURN_PORTS:-}
|
||||
MUMH5_TURN_URLS: ${MUMH5_TURN_URLS:-}
|
||||
MUMH5_DEBUG: ${MUMH5_DEBUG:-}
|
||||
MUMH5_TURN_MAX: ${MUMH5_TURN_MAX:-}
|
||||
MUMH5_TURN_PER_ADDRESS: ${MUMH5_TURN_PER_ADDRESS:-}
|
||||
MUMH5_MAX_CONNECTIONS: ${MUMH5_MAX_CONNECTIONS:-}
|
||||
MUMH5_MAX_PER_ADDRESS: ${MUMH5_MAX_PER_ADDRESS:-}
|
||||
+66
-2
@@ -1,4 +1,4 @@
|
||||
import { app, BrowserWindow, dialog, ipcMain, net, shell, session, systemPreferences, type WebContents } from 'electron';
|
||||
import { app, BrowserWindow, desktopCapturer, dialog, ipcMain, net, shell, session, systemPreferences, type WebContents } from 'electron';
|
||||
import { promises as fs } from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
@@ -9,9 +9,15 @@ import { fetchLinkPreview } from './link-preview.ts';
|
||||
import { fetchPublicListWith, pingServer } from './publist.ts';
|
||||
import { openTls } from './tls-transport.ts';
|
||||
import { udpChannel } from './udp-voice.ts';
|
||||
import { ShareAudio, pipewireAvailable, VIRTUAL_MIC_LABEL, type AudioTarget } from './pipewire.ts';
|
||||
|
||||
const devUrl = process.env.VITE_DEV_SERVER_URL;
|
||||
|
||||
// Screen sharing connects mumh5 users directly. Chromium would hide this computer's addresses
|
||||
// behind names that only resolve on the local network, so two people on IPv6 could not reach
|
||||
// each other without a helper server. Only this app's own page runs here.
|
||||
app.commandLine.appendSwitch('disable-features', 'WebRtcHideLocalIpsWithMdns');
|
||||
|
||||
let identityStore: IdentityStore | null = null;
|
||||
const identities = () => (identityStore ??= new IdentityStore(app.getPath('userData')));
|
||||
|
||||
@@ -120,6 +126,49 @@ ipcMain.handle('publist:ping', (_e, host: string, port: number) => pingServer(St
|
||||
|
||||
ipcMain.handle('preview:fetch', (_e, url: string) => fetchLinkPreview(String(url).slice(0, 2048)));
|
||||
|
||||
// ─── Screen sharing ───────────────────────────────────────────────────────────
|
||||
// The renderer lists the sources, the user picks one, and the next getDisplayMedia call gets it.
|
||||
// On Wayland, listing already opens the system's picker and returns the one chosen source;
|
||||
// `chosen` tells the renderer not to ask a second time.
|
||||
let screenSources: Electron.DesktopCapturerSource[] = [];
|
||||
let pickedSource: string | null = null;
|
||||
const onWayland = () => process.platform === 'linux' && app.commandLine.getSwitchValue('ozone-platform') !== 'x11' &&
|
||||
(process.env.XDG_SESSION_TYPE === 'wayland' || !!process.env.WAYLAND_DISPLAY);
|
||||
|
||||
// Sound: PipeWire on Linux (one program or everything but this app), system loopback on Windows
|
||||
const shareAudio = new ShareAudio(() => app.getAppMetrics().map(m => m.pid));
|
||||
ipcMain.handle('share:info', async () => ({
|
||||
portal: onWayland(),
|
||||
audio: process.platform === 'win32' ? 'loopback' : (await pipewireAvailable()) ? 'pipewire' : 'none',
|
||||
micLabel: VIRTUAL_MIC_LABEL
|
||||
}));
|
||||
ipcMain.handle('share:audioApps', () => shareAudio.apps().catch(() => []));
|
||||
ipcMain.handle('share:audioStart', (_e, target: AudioTarget) =>
|
||||
shareAudio.start(target?.kind === 'app' ? { kind: 'app', name: String(target.name) } : { kind: 'all' }));
|
||||
ipcMain.handle('share:audioStop', () => shareAudio.stop());
|
||||
// The virtual microphone outlives the process otherwise
|
||||
app.on('will-quit', () => { shareAudio.stop(); });
|
||||
|
||||
// Names first (fast and dependable), thumbnails in a second call: grabbing the pictures can be
|
||||
// slow and, under X11, can come back with nothing.
|
||||
ipcMain.handle('share:sources', async (_e, thumbnails: boolean) => {
|
||||
const wayland = onWayland();
|
||||
const size = thumbnails ? { width: 320, height: 180 } : { width: 0, height: 0 };
|
||||
const list = () => desktopCapturer.getSources({ types: ['screen', 'window'], thumbnailSize: size });
|
||||
let found = await list();
|
||||
// Not on Wayland, where empty means the user cancelled the system's picker and asking again would reopen it
|
||||
for (let i = 0; i < 3 && !found.length && !wayland; i++) {
|
||||
await new Promise(r => setTimeout(r, 250));
|
||||
found = await list();
|
||||
}
|
||||
if (found.length || !thumbnails) screenSources = found;
|
||||
return {
|
||||
chosen: wayland && found.length === 1,
|
||||
sources: found.map(s => ({ id: s.id, name: s.name, thumbnail: s.thumbnail.isEmpty() ? '' : s.thumbnail.toDataURL() }))
|
||||
};
|
||||
});
|
||||
ipcMain.on('share:pick', (_e, id: string) => { pickedSource = String(id); });
|
||||
|
||||
ipcMain.on('tray:update', (_e, state: tray.TrayState) => tray.update(state));
|
||||
|
||||
ipcMain.handle('platform:info', () => ({ os: process.platform, osVersion: os.release() }));
|
||||
@@ -152,7 +201,11 @@ function createWindow() {
|
||||
});
|
||||
|
||||
// Links from chat open in the system browser, never inside the app
|
||||
win.webContents.setWindowOpenHandler(({ url }) => {
|
||||
win.webContents.setWindowOpenHandler(({ url, frameName }) => {
|
||||
// The stream window: an empty page the app fills with the video itself
|
||||
if (frameName === 'mumh5-stream' && (url === 'about:blank' || url === '')) {
|
||||
return { action: 'allow', overrideBrowserWindowOptions: { width: 960, height: 600, minWidth: 320, minHeight: 200, backgroundColor: '#000000', autoHideMenuBar: true, title: 'mumh5' } };
|
||||
}
|
||||
if (/^https?:\/\//i.test(url)) shell.openExternal(url);
|
||||
return { action: 'deny' };
|
||||
});
|
||||
@@ -195,6 +248,17 @@ app.whenReady().then(async () => {
|
||||
cb({ requestHeaders: details.requestHeaders });
|
||||
});
|
||||
|
||||
session.defaultSession.setDisplayMediaRequestHandler((request, callback) => {
|
||||
const source = screenSources.find(s => s.id === pickedSource);
|
||||
pickedSource = null;
|
||||
// Without a picked source the request is refused
|
||||
try {
|
||||
if (!source) callback({});
|
||||
else if (request.audioRequested && process.platform === 'win32') callback({ video: source, audio: 'loopback' });
|
||||
else callback({ video: source });
|
||||
} catch { /* refused */ }
|
||||
});
|
||||
|
||||
// Microphone, camera and screen capture for voice and video; nothing else
|
||||
session.defaultSession.setPermissionRequestHandler((_wc, permission, cb) => {
|
||||
cb(['media', 'display-capture', 'clipboard-sanitized-write', 'notifications'].includes(permission));
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
// Sound for screen sharing on Linux. Chromium cannot capture what other programs play, so a
|
||||
// virtual microphone is created in PipeWire and the chosen programs' output is linked into it;
|
||||
// the renderer then records that microphone like any other. Works through PipeWire's command
|
||||
// line tools (pw-cli, pw-dump, pw-link, pw-mon), no native module.
|
||||
import { execFile, spawn, type ChildProcess } from 'node:child_process';
|
||||
|
||||
export const VIRTUAL_MIC = 'mumh5-share-audio';
|
||||
export const VIRTUAL_MIC_LABEL = 'mumh5 screen share audio';
|
||||
|
||||
// What to capture: everything except this app's own sound, or one program
|
||||
export type AudioTarget = { kind: 'all' } | { kind: 'app'; name: string };
|
||||
|
||||
interface PwObject { id: number; type: string; info?: { direction?: string; props?: Record<string, any> } }
|
||||
|
||||
function run(cmd: string, args: string[], maxBuffer = 64 * 1024 * 1024): Promise<string> {
|
||||
return new Promise((resolve, reject) => {
|
||||
execFile(cmd, args, { maxBuffer, timeout: 5000 }, (err, stdout) => err ? reject(err) : resolve(stdout));
|
||||
});
|
||||
}
|
||||
|
||||
const dump = async (): Promise<PwObject[]> => JSON.parse(await run('pw-dump', []));
|
||||
|
||||
// A playing program: its name and process come from the stream itself or from the client that owns it
|
||||
function describe(node: PwObject, byId: Map<number, PwObject>): { name: string; pid: number | null } {
|
||||
const p = node.info?.props ?? {};
|
||||
const client = byId.get(p['client.id'])?.info?.props ?? {};
|
||||
return {
|
||||
name: String(p['application.name'] ?? client['application.name'] ?? p['node.name'] ?? ''),
|
||||
pid: Number(p['application.process.id'] ?? client['application.process.id']) || null
|
||||
};
|
||||
}
|
||||
|
||||
const outputs = (objects: PwObject[]) =>
|
||||
objects.filter(o => o.type === 'PipeWire:Interface:Node' && o.info?.props?.['media.class'] === 'Stream/Output/Audio');
|
||||
|
||||
// Programs currently playing sound, without this app
|
||||
export function audioApps(objects: PwObject[], ownPids: number[]): string[] {
|
||||
const byId = new Map(objects.map(o => [o.id, o]));
|
||||
const names = new Set<string>();
|
||||
for (const node of outputs(objects)) {
|
||||
const d = describe(node, byId);
|
||||
if (d.name && !(d.pid && ownPids.includes(d.pid))) names.add(d.name);
|
||||
}
|
||||
return [...names].sort((a, b) => a.localeCompare(b));
|
||||
}
|
||||
|
||||
// Which output ports to connect to which inputs of the virtual microphone, and which existing
|
||||
// links to remove. This app's own playback is never linked: the people watching are in the same
|
||||
// voice channel and would hear themselves back.
|
||||
export function planLinks(objects: PwObject[], target: AudioTarget, ownPids: number[]): { add: [number, number][]; remove: number[] } {
|
||||
const byId = new Map(objects.map(o => [o.id, o]));
|
||||
const mic = objects.find(o => o.type === 'PipeWire:Interface:Node' && o.info?.props?.['node.name'] === VIRTUAL_MIC);
|
||||
if (!mic) return { add: [], remove: [] };
|
||||
const ports = objects.filter(o => o.type === 'PipeWire:Interface:Port');
|
||||
const micIn = ports.filter(p => p.info?.props?.['node.id'] === mic.id && p.info?.direction === 'input');
|
||||
const input = (channel: string) => micIn.find(p => p.info?.props?.['audio.channel'] === channel)?.id;
|
||||
const left = input('FL'), right = input('FR');
|
||||
if (left == null || right == null) return { add: [], remove: [] };
|
||||
|
||||
const wanted = new Set<string>();
|
||||
for (const node of outputs(objects)) {
|
||||
const d = describe(node, byId);
|
||||
if (d.pid && ownPids.includes(d.pid)) continue;
|
||||
if (target.kind === 'app' && d.name !== target.name) continue;
|
||||
for (const port of ports) {
|
||||
if (port.info?.props?.['node.id'] !== node.id || port.info?.direction !== 'output') continue;
|
||||
const channel = port.info.props['audio.channel'];
|
||||
// Mono programs go to both sides; surround beyond the front pair is left out
|
||||
if (channel === 'FL' || channel === 'MONO') wanted.add(`${port.id}:${left}`);
|
||||
if (channel === 'FR' || channel === 'MONO') wanted.add(`${port.id}:${right}`);
|
||||
}
|
||||
}
|
||||
const remove: number[] = [];
|
||||
for (const link of objects.filter(o => o.type === 'PipeWire:Interface:Link')) {
|
||||
const info = link.info as any;
|
||||
if (info?.['input-node-id'] !== mic.id) continue;
|
||||
const key = `${info['output-port-id']}:${info['input-port-id']}`;
|
||||
if (wanted.has(key)) wanted.delete(key);
|
||||
else remove.push(link.id);
|
||||
}
|
||||
return { add: [...wanted].map(k => k.split(':').map(Number) as [number, number]), remove };
|
||||
}
|
||||
|
||||
// Whether PipeWire and its tools are there
|
||||
export async function pipewireAvailable(): Promise<boolean> {
|
||||
if (process.platform !== 'linux') return false;
|
||||
try { await dump(); await run('pw-link', ['--version']); await run('pw-cli', ['--version']); return true; } catch { return false; }
|
||||
}
|
||||
|
||||
export class ShareAudio {
|
||||
private target: AudioTarget | null = null;
|
||||
private monitor: ChildProcess | null = null;
|
||||
private debounce: ReturnType<typeof setTimeout> | undefined;
|
||||
private busy: Promise<void> = Promise.resolve();
|
||||
private ownPids: () => number[];
|
||||
|
||||
constructor(ownPids: () => number[]) { this.ownPids = ownPids; }
|
||||
|
||||
async apps(): Promise<string[]> {
|
||||
return audioApps(await dump(), this.ownPids());
|
||||
}
|
||||
|
||||
// Creates the microphone, links the target and keeps the links right while programs come and go
|
||||
async start(target: AudioTarget): Promise<void> {
|
||||
this.target = target;
|
||||
if (!(await dump()).some(o => o.info?.props?.['node.name'] === VIRTUAL_MIC)) {
|
||||
await run('pw-cli', ['create-node', 'adapter',
|
||||
`{ factory.name=support.null-audio-sink node.name=${VIRTUAL_MIC} node.description="${VIRTUAL_MIC_LABEL}" media.class=Audio/Source/Virtual object.linger=1 audio.position=[FL,FR] audio.rate=48000 audio.channels=2 }`]);
|
||||
// Give PipeWire a moment to publish the node and its ports
|
||||
for (let i = 0; i < 20; i++) {
|
||||
const objects = await dump();
|
||||
const mic = objects.find(o => o.info?.props?.['node.name'] === VIRTUAL_MIC);
|
||||
if (mic && objects.some(o => o.type === 'PipeWire:Interface:Port' && o.info?.props?.['node.id'] === mic.id)) break;
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
}
|
||||
}
|
||||
await this.relink();
|
||||
if (!this.monitor) {
|
||||
const proc = spawn('pw-mon', ['--color=never'], { stdio: ['ignore', 'pipe', 'ignore'] });
|
||||
this.monitor = proc;
|
||||
proc.stdout!.on('data', (data: Buffer) => {
|
||||
const text = data.toString();
|
||||
if (!text.includes('added') && !text.includes('removed')) return;
|
||||
// Starting a program fires many events at once
|
||||
clearTimeout(this.debounce);
|
||||
this.debounce = setTimeout(() => this.relink().catch(() => {}), 400);
|
||||
});
|
||||
proc.on('error', () => { if (this.monitor === proc) this.monitor = null; });
|
||||
proc.on('exit', () => { if (this.monitor === proc) this.monitor = null; });
|
||||
}
|
||||
}
|
||||
|
||||
// One at a time, so two graph changes cannot interleave their link commands
|
||||
private relink(): Promise<void> {
|
||||
const next = this.busy.then(async () => {
|
||||
if (!this.target) return;
|
||||
const { add, remove } = planLinks(await dump(), this.target, this.ownPids());
|
||||
for (const id of remove) await run('pw-link', ['-d', String(id)]).catch(() => {});
|
||||
// "File exists" when a link is already there is fine
|
||||
for (const [out, input] of add) await run('pw-link', [String(out), String(input)]).catch(() => {});
|
||||
});
|
||||
this.busy = next.catch(() => {});
|
||||
return next;
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
this.target = null;
|
||||
clearTimeout(this.debounce);
|
||||
this.monitor?.kill();
|
||||
this.monitor = null;
|
||||
await this.busy;
|
||||
try {
|
||||
for (const o of await dump()) {
|
||||
if (o.type === 'PipeWire:Interface:Node' && o.info?.props?.['node.name'] === VIRTUAL_MIC) await run('pw-cli', ['destroy', String(o.id)]).catch(() => {});
|
||||
}
|
||||
} catch { /* PipeWire gone */ }
|
||||
}
|
||||
}
|
||||
@@ -21,6 +21,12 @@ contextBridge.exposeInMainWorld('mumh5Native', {
|
||||
onContextMenu: (fn: (params: unknown) => void) => { ipcRenderer.on('context-menu', (_e, p) => fn(p)); },
|
||||
editAction: (action: string, arg?: unknown) => ipcRenderer.send('edit:action', action, arg),
|
||||
onTrayAction: (fn: (action: string) => void) => { ipcRenderer.on('tray:action', (_e, a: string) => fn(a)); },
|
||||
screenSources: (thumbnails: boolean) => ipcRenderer.invoke('share:sources', !!thumbnails),
|
||||
pickScreenSource: (id: string) => ipcRenderer.send('share:pick', id),
|
||||
shareInfo: () => ipcRenderer.invoke('share:info'),
|
||||
shareAudioApps: () => ipcRenderer.invoke('share:audioApps'),
|
||||
shareAudioStart: (target: unknown) => ipcRenderer.invoke('share:audioStart', target),
|
||||
shareAudioStop: () => ipcRenderer.invoke('share:audioStop'),
|
||||
describeCerts: (ders: Uint8Array[]) => ipcRenderer.invoke('certs:describe', ders),
|
||||
identities: {
|
||||
list: () => ipcRenderer.invoke('identities:list'),
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: https: http:; media-src 'self' blob: https: http:; connect-src 'self' https: http: ws: wss:; object-src 'none'; base-uri 'none'; frame-src https://www.youtube-nocookie.com" />
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' 'wasm-unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: https: http:; media-src 'self' blob: https: http:; connect-src 'self' https: http: ws: wss:; object-src 'none'; base-uri 'none'; frame-src https://www.youtube-nocookie.com" />
|
||||
<title>mumh5</title>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
Generated
+8
@@ -14,6 +14,7 @@
|
||||
"protobufjs": "^8.8.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@evan/opus": "^1.0.3",
|
||||
"@sveltejs/vite-plugin-svelte": "^7.3.1",
|
||||
"@tsconfig/svelte": "^5.0.8",
|
||||
"@types/node": "^26.6.3",
|
||||
@@ -827,6 +828,13 @@
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/@evan/opus": {
|
||||
"version": "1.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@evan/opus/-/opus-1.0.3.tgz",
|
||||
"integrity": "sha512-ADfwIad83W1LuiZDNMjDMDNQRsPz8rj5xnDLExhVWTnA5wGJCLntOn12Ir5rxGBqdfo10QhnNVdd2+gXiZ6xCg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@isaacs/fs-minipass": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
"dist:mac": "npm run build && electron-builder --mac",
|
||||
"test:e2e": "node test/e2e/app.e2e.ts",
|
||||
"test:e2e:web": "node test/e2e/web.e2e.ts",
|
||||
"test:e2e:share": "node test/e2e/share.e2e.ts",
|
||||
"proto": "pbjs -t static-module -w es6 --keep-case --no-delimited --no-service --no-comments --force-number proto/Mumble.proto -o src/core/mumble-pb.js && pbjs -t static-module -w es6 --keep-case --no-delimited --no-service --no-comments --force-number proto/MumbleUDP.proto -o src/core/mumble-udp-pb.js",
|
||||
"screenshots": "node test/e2e/screenshots.ts"
|
||||
},
|
||||
@@ -89,6 +90,7 @@
|
||||
}
|
||||
},
|
||||
"devDependencies": {
|
||||
"@evan/opus": "^1.0.3",
|
||||
"@sveltejs/vite-plugin-svelte": "^7.3.1",
|
||||
"@tsconfig/svelte": "^5.0.8",
|
||||
"@types/node": "^26.6.3",
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ import { createServer } from 'vite';
|
||||
// The browser build for development: the web proxy on 127.0.0.1:8080 and the Vite dev server,
|
||||
// which forwards /api to it (vite.config.ts). Without MUMH5_SERVERS the proxy allows any
|
||||
// server, private addresses included; it only listens on this machine.
|
||||
const env = { ...process.env, MUMH5_PORT: '8080', MUMH5_BIND: '127.0.0.1' };
|
||||
const env = { MUMH5_STUN_BIND: '127.0.0.1', ...process.env, MUMH5_PORT: '8080', MUMH5_BIND: '127.0.0.1' };
|
||||
if (!env.MUMH5_SERVERS && !env.MUMH5_ALLOW_ANY) Object.assign(env, { MUMH5_ALLOW_ANY: '1', MUMH5_ALLOW_PRIVATE: '1' });
|
||||
|
||||
const proxy = spawn(process.execPath, ['server/main.ts'], { stdio: 'inherit', env });
|
||||
|
||||
+4
-1
@@ -8,10 +8,13 @@ const config = configFromEnv(process.env);
|
||||
config.staticDir ??= [path.resolve(import.meta.dirname, '../dist-web')].find(d => existsSync(path.join(d, 'index.html'))) ?? null;
|
||||
|
||||
try {
|
||||
const { port } = await startProxy(config);
|
||||
const { port, stunPort } = await startProxy(config);
|
||||
console.log(`mumh5 proxy listening on http://${config.bind}:${port}`);
|
||||
console.log(config.allowAny ? `Allowed servers: any${config.allowPrivate ? ', private addresses included' : ' public address'}` : `Allowed servers: ${config.servers.map(s => `${s.host}:${s.port}`).join(', ')}`);
|
||||
if (config.sendProxy) console.log('Announcing client addresses with the PROXY protocol; the allowed servers must expect it');
|
||||
console.log(!stunPort ? 'STUN is off; screen sharing between browser users will only work on the same network'
|
||||
: config.turn ? `Screen sharing: STUN and relay on port ${stunPort} (UDP and TCP), relayed streams on UDP ${config.turnMinPort}-${config.turnMaxPort}; all must be reachable from the internet`
|
||||
: `Screen sharing: STUN on UDP port ${stunPort} (must be reachable from the internet), no relay`);
|
||||
console.log(config.staticDir ? `Serving the web app from ${config.staticDir}` : 'No web build found (npm run build:web); serving the API only');
|
||||
} catch (e) {
|
||||
console.error((e as Error).message);
|
||||
|
||||
+77
-18
@@ -4,12 +4,16 @@
|
||||
import http from 'node:http';
|
||||
import dns from 'node:dns';
|
||||
import net from 'node:net';
|
||||
import { randomBytes } from 'node:crypto';
|
||||
import path from 'node:path';
|
||||
import { promises as fs } from 'node:fs';
|
||||
import { WebSocketServer, type WebSocket } from 'ws';
|
||||
import { openTls } from '../electron/tls-transport.ts';
|
||||
import { generateIdentity, identityFromP12, identityToP12, certCommonName } from '../electron/identity.ts';
|
||||
import { describeCert } from '../electron/certs.ts';
|
||||
import { startTurn, turnCredential, stunResponse, isPrivateAddress, CREDENTIAL_TTL } from './turn.ts';
|
||||
|
||||
export { stunResponse, isPrivateAddress };
|
||||
|
||||
export interface AllowedServer { host: string; port: number; label: string }
|
||||
|
||||
@@ -29,13 +33,30 @@ export interface ProxyConfig {
|
||||
// Only for servers behind something that understands it (go-mmproxy); plain Mumble does not.
|
||||
sendProxy: boolean;
|
||||
staticDir: string | null;
|
||||
// UDP port of the built-in STUN responder for screen sharing between browser users; null turns it off
|
||||
stunPort: number | null;
|
||||
stunBind: string;
|
||||
// Relay streams for browsers that cannot connect directly (TURN, same port over UDP and TCP)
|
||||
turn: boolean;
|
||||
// Public address announced for relayed traffic, when it cannot be found (behind a 1:1 NAT)
|
||||
turnIp: string | null;
|
||||
turnMinPort: number;
|
||||
turnMaxPort: number;
|
||||
// Relay routes in total and per client address. A browser opens one per stream and network
|
||||
// interface, and a camera round has a stream per pair of people, so these run high.
|
||||
turnMax: number;
|
||||
turnPerAddress: number;
|
||||
// Addresses to hand to clients instead of this host's port, e.g. a TLS front on port 443
|
||||
turnUrls: string[];
|
||||
// Log relay events
|
||||
debug: boolean;
|
||||
maxConnections: number;
|
||||
maxPerAddress: number;
|
||||
}
|
||||
|
||||
export const defaults: ProxyConfig = {
|
||||
port: 8080, bind: '127.0.0.1', servers: [], allowAny: false, allowPrivate: false, origins: [],
|
||||
trustProxy: false, sendProxy: false, staticDir: null, maxConnections: 200, maxPerAddress: 8
|
||||
trustProxy: false, sendProxy: false, staticDir: null, stunPort: null, stunBind: '::', turn: true, turnIp: null, turnMinPort: 49160, turnMaxPort: 49659, turnMax: 500, turnPerAddress: 64, turnUrls: [], debug: false, maxConnections: 200, maxPerAddress: 8
|
||||
};
|
||||
|
||||
// "host", "host:port", "[v6]:port", each optionally followed by "=Label"
|
||||
@@ -51,7 +72,9 @@ export function parseServers(list: string): AllowedServer[] {
|
||||
});
|
||||
}
|
||||
|
||||
export function configFromEnv(env: NodeJS.ProcessEnv): ProxyConfig {
|
||||
export function configFromEnv(source: NodeJS.ProcessEnv): ProxyConfig {
|
||||
// A variable set to nothing counts as not set (docker-compose passes unset ones on that way)
|
||||
const env = Object.fromEntries(Object.entries(source).filter(([, v]) => v !== '')) as NodeJS.ProcessEnv;
|
||||
const on = (v?: string) => v === '1' || v === 'true';
|
||||
return {
|
||||
...defaults,
|
||||
@@ -64,24 +87,21 @@ export function configFromEnv(env: NodeJS.ProcessEnv): ProxyConfig {
|
||||
trustProxy: on(env.MUMH5_TRUST_PROXY),
|
||||
sendProxy: on(env.MUMH5_SEND_PROXY),
|
||||
staticDir: env.MUMH5_STATIC ?? null,
|
||||
stunPort: Number(env.MUMH5_STUN_PORT ?? 3478) || null,
|
||||
stunBind: env.MUMH5_STUN_BIND ?? defaults.stunBind,
|
||||
turn: env.MUMH5_TURN !== '0' && env.MUMH5_TURN !== 'false',
|
||||
turnIp: env.MUMH5_TURN_IP ?? null,
|
||||
turnMinPort: Number((env.MUMH5_TURN_PORTS ?? '').split('-')[0]) || defaults.turnMinPort,
|
||||
turnMaxPort: Number((env.MUMH5_TURN_PORTS ?? '').split('-')[1]) || defaults.turnMaxPort,
|
||||
turnMax: Number(env.MUMH5_TURN_MAX ?? defaults.turnMax),
|
||||
turnPerAddress: Number(env.MUMH5_TURN_PER_ADDRESS ?? defaults.turnPerAddress),
|
||||
turnUrls: (env.MUMH5_TURN_URLS ?? '').split(',').map(s => s.trim()).filter(Boolean),
|
||||
debug: on(env.MUMH5_DEBUG),
|
||||
maxConnections: Number(env.MUMH5_MAX_CONNECTIONS ?? defaults.maxConnections),
|
||||
maxPerAddress: Number(env.MUMH5_MAX_PER_ADDRESS ?? defaults.maxPerAddress)
|
||||
};
|
||||
}
|
||||
|
||||
// Loopback, private, link-local and other addresses that are not on the public internet
|
||||
export function isPrivateAddress(address: string): boolean {
|
||||
if (net.isIPv4(address)) {
|
||||
const [a, b] = address.split('.').map(Number);
|
||||
return a === 0 || a === 10 || a === 127 || (a === 100 && b >= 64 && b <= 127) || (a === 169 && b === 254) ||
|
||||
(a === 172 && b >= 16 && b <= 31) || (a === 192 && b === 168) || a >= 224;
|
||||
}
|
||||
const v6 = address.toLowerCase();
|
||||
const mapped = /^::ffff:(\d+\.\d+\.\d+\.\d+)$/.exec(v6);
|
||||
if (mapped) return isPrivateAddress(mapped[1]);
|
||||
return v6 === '::' || v6 === '::1' || /^f[cd]/.test(v6) || /^fe[89ab]/.test(v6) || /^ff/.test(v6);
|
||||
}
|
||||
|
||||
// DNS lookup that fails for private addresses, so a public name cannot point the proxy inward
|
||||
const publicLookup: net.LookupFunction = (hostname, options, callback) => {
|
||||
// With `all` the result is a list of addresses, otherwise one address string
|
||||
@@ -110,12 +130,15 @@ class HttpError extends Error {
|
||||
constructor(status: number, message: string) { super(message); this.status = status; }
|
||||
}
|
||||
|
||||
export async function startProxy(config: ProxyConfig): Promise<{ port: number; close(): Promise<void> }> {
|
||||
export async function startProxy(config: ProxyConfig): Promise<{ port: number; stunPort: number | null; close(): Promise<void> }> {
|
||||
if (!config.allowAny && !config.servers.length) {
|
||||
throw new Error('No servers allowed. Set MUMH5_SERVERS=host[:port][=Label],... or MUMH5_ALLOW_ANY=1.');
|
||||
}
|
||||
const staticDir = config.staticDir ? path.resolve(config.staticDir) : null;
|
||||
const perAddress = new Map<string, number>();
|
||||
let stunPort: number | null = null;
|
||||
// Made up at every start: credentials are handed out by this process and checked by it
|
||||
const turnSecret = randomBytes(24).toString('hex');
|
||||
// Identity requests per address in the current minute; key generation is the costly part
|
||||
const identityUse = new Map<string, number>();
|
||||
const sweep = setInterval(() => identityUse.clear(), 60000);
|
||||
@@ -154,7 +177,22 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; c
|
||||
|
||||
async function api(req: http.IncomingMessage, route: string): Promise<unknown> {
|
||||
if (route === 'config' && req.method === 'GET') {
|
||||
return { servers: config.servers, any: config.allowAny };
|
||||
return { servers: config.servers, any: config.allowAny, stun: stunPort, turn: stunPort != null && config.turn };
|
||||
}
|
||||
// What a client needs for screen sharing: the STUN port and, with the relay on, credentials
|
||||
// for it. Open to any origin, because the desktop app asks from outside this site; checking
|
||||
// the origin would not keep a script out anyway. The quotas are what limit use.
|
||||
if (route === 'relay' && req.method === 'GET') {
|
||||
if (stunPort == null) throw new HttpError(404, 'No STUN or relay here');
|
||||
if (!config.turn) return { port: stunPort };
|
||||
const addr = addressOf(req);
|
||||
const used = (identityUse.get(addr) ?? 0) + 1;
|
||||
identityUse.set(addr, used);
|
||||
if (used > 20) throw new HttpError(429, 'Too many requests, try again in a minute');
|
||||
// The username is the time it runs out; the relay recomputes the password from it
|
||||
const username = String(Math.floor(Date.now() / 1000) + CREDENTIAL_TTL);
|
||||
if (config.debug) console.log(new Date().toISOString(), `relay: credentials for ${addr}`);
|
||||
return { port: stunPort, username, credential: turnCredential(turnSecret, username), ttl: CREDENTIAL_TTL, ...(config.turnUrls.length ? { urls: config.turnUrls } : {}) };
|
||||
}
|
||||
if (req.method !== 'POST') throw new HttpError(404, 'Not found');
|
||||
if (!originOk(req)) throw new HttpError(403, 'Origin not allowed');
|
||||
@@ -213,7 +251,11 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; c
|
||||
const work = pathname.startsWith('/api/')
|
||||
? api(req, pathname.slice(5)).then(result => {
|
||||
const data = Buffer.from(JSON.stringify(result));
|
||||
res.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': data.length, 'Cache-Control': 'no-store', 'X-Content-Type-Options': 'nosniff' });
|
||||
res.writeHead(200, {
|
||||
'Content-Type': 'application/json', 'Content-Length': data.length, 'Cache-Control': 'no-store', 'X-Content-Type-Options': 'nosniff',
|
||||
// The desktop app reads this one from another origin
|
||||
...(pathname === '/api/relay' ? { 'Access-Control-Allow-Origin': '*' } : {})
|
||||
});
|
||||
res.end(data);
|
||||
})
|
||||
: serveStatic(req, res, pathname);
|
||||
@@ -305,10 +347,27 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; c
|
||||
server.once('error', reject);
|
||||
server.listen(config.port, config.bind, resolve);
|
||||
});
|
||||
|
||||
// STUN and the relay. Failing to open the port only costs screen sharing its helper.
|
||||
let turn: { port: number; close(): void } | null = null;
|
||||
if (config.stunPort != null) {
|
||||
try {
|
||||
turn = await startTurn({
|
||||
port: config.stunPort, bind: config.stunBind, relay: config.turn, publicIp: config.turnIp,
|
||||
minPort: config.turnMinPort, maxPort: config.turnMaxPort, maxAllocations: config.turnMax, maxPerAddress: config.turnPerAddress,
|
||||
peerAllowed: ip => config.allowPrivate || !isPrivateAddress(ip),
|
||||
log: config.debug ? line => console.log(new Date().toISOString(), line) : undefined
|
||||
}, turnSecret);
|
||||
stunPort = turn.port;
|
||||
} catch { /* port taken or not permitted */ }
|
||||
}
|
||||
|
||||
return {
|
||||
port: (server.address() as net.AddressInfo).port,
|
||||
stunPort,
|
||||
close: () => new Promise<void>(resolve => {
|
||||
clearInterval(sweep);
|
||||
turn?.close();
|
||||
for (const ws of wss.clients) ws.terminate();
|
||||
wss.close();
|
||||
server.close(() => resolve());
|
||||
|
||||
+406
@@ -0,0 +1,406 @@
|
||||
// STUN and TURN for screen sharing between browser users (RFC 5389, RFC 5766).
|
||||
// STUN tells a browser its public address so two of them can connect directly. Where that is
|
||||
// not possible (mobile carriers, company networks, browsers that forbid direct UDP such as
|
||||
// Vanadium) the stream goes through this relay instead: the browser reaches it over UDP or
|
||||
// TCP, and the relay passes the packets on over UDP. Credentials are short-lived and come
|
||||
// from the proxy's API, so only people using this site can relay through it.
|
||||
import dgram from 'node:dgram';
|
||||
import net from 'node:net';
|
||||
import os from 'node:os';
|
||||
import { createHash, createHmac, randomBytes, timingSafeEqual } from 'node:crypto';
|
||||
|
||||
const COOKIE = 0x2112a442;
|
||||
const REALM = 'mumh5';
|
||||
|
||||
// Message types: request, with success (| 0x100) and error (| 0x110) answers
|
||||
const BINDING = 0x0001, ALLOCATE = 0x0003, REFRESH = 0x0004, SEND = 0x0016, DATA = 0x0017, CREATE_PERMISSION = 0x0008, CHANNEL_BIND = 0x0009;
|
||||
const A = {
|
||||
USERNAME: 0x0006, MESSAGE_INTEGRITY: 0x0008, ERROR_CODE: 0x0009, CHANNEL_NUMBER: 0x000c, LIFETIME: 0x000d, XOR_PEER_ADDRESS: 0x0012,
|
||||
DATA: 0x0013, REALM: 0x0014, NONCE: 0x0015, XOR_RELAYED_ADDRESS: 0x0016, REQUESTED_ADDRESS_FAMILY: 0x0017, REQUESTED_TRANSPORT: 0x0019,
|
||||
XOR_MAPPED_ADDRESS: 0x0020
|
||||
};
|
||||
|
||||
const LIFETIME = 600; // an allocation, unless refreshed
|
||||
const PERMISSION = 300; // a peer address may send for this long after CreatePermission
|
||||
const CHANNEL = 600;
|
||||
const NONCE_AGE = 3600;
|
||||
export const CREDENTIAL_TTL = 12 * 3600;
|
||||
|
||||
export interface TurnOptions {
|
||||
port: number;
|
||||
bind: string;
|
||||
// false: answer STUN binding requests only
|
||||
relay: boolean;
|
||||
// Address announced for relayed traffic; found from the connection or the network interfaces when unset
|
||||
publicIp: string | null;
|
||||
// UDP ports used for relaying, so a firewall can open exactly these
|
||||
minPort: number;
|
||||
maxPort: number;
|
||||
maxAllocations: number;
|
||||
maxPerAddress: number;
|
||||
// Peers the relay may talk to; keeps it from being used to reach private networks
|
||||
peerAllowed: (ip: string) => boolean;
|
||||
// One line per relay event, for finding out why someone cannot connect
|
||||
log?: (line: string) => void;
|
||||
}
|
||||
|
||||
// Loopback, private, link-local and other addresses that are not on the public internet
|
||||
export function isPrivateAddress(address: string): boolean {
|
||||
if (net.isIPv4(address)) {
|
||||
const [a, b] = address.split('.').map(Number);
|
||||
return a === 0 || a === 10 || a === 127 || (a === 100 && b >= 64 && b <= 127) || (a === 169 && b === 254) ||
|
||||
(a === 172 && b >= 16 && b <= 31) || (a === 192 && b === 168) || a >= 224;
|
||||
}
|
||||
const v6 = address.toLowerCase();
|
||||
const mapped = /^::ffff:(\d+\.\d+\.\d+\.\d+)$/.exec(v6);
|
||||
if (mapped) return isPrivateAddress(mapped[1]);
|
||||
return v6 === '::' || v6 === '::1' || /^f[cd]/.test(v6) || /^fe[89ab]/.test(v6) || /^ff/.test(v6);
|
||||
}
|
||||
|
||||
const v4 = (address: string) => address.replace(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/i, '$1');
|
||||
|
||||
function ipBytes(address: string): number[] | null {
|
||||
const a = v4(address);
|
||||
if (net.isIPv4(a)) return a.split('.').map(Number);
|
||||
if (!net.isIPv6(a)) return null;
|
||||
const [head, tail = ''] = a.split('%')[0].split('::');
|
||||
const h = head ? head.split(':') : [], t = tail ? tail.split(':') : [];
|
||||
const groups = a.includes('::') ? [...h, ...new Array(8 - h.length - t.length).fill('0'), ...t] : h;
|
||||
return groups.flatMap(g => { const n = parseInt(g, 16); return [n >> 8, n & 255]; });
|
||||
}
|
||||
|
||||
// Address attribute value, masked with the cookie and (for IPv6) the transaction id
|
||||
export function xorAddress(address: string, port: number, header: Uint8Array): Uint8Array | null {
|
||||
const bytes = ipBytes(address);
|
||||
if (!bytes) return null;
|
||||
const out = new Uint8Array(4 + bytes.length);
|
||||
out[1] = bytes.length === 4 ? 1 : 2;
|
||||
out[2] = (port >> 8) ^ 0x21;
|
||||
out[3] = (port & 255) ^ 0x12;
|
||||
for (let i = 0; i < bytes.length; i++) out[4 + i] = bytes[i] ^ header[4 + i];
|
||||
return out;
|
||||
}
|
||||
|
||||
export function unxorAddress(value: Uint8Array, header: Uint8Array): { ip: string; port: number } | null {
|
||||
if (value.length !== 8 && value.length !== 20) return null;
|
||||
const port = ((value[2] ^ 0x21) << 8) | (value[3] ^ 0x12);
|
||||
const raw = [...value.subarray(4)].map((b, i) => b ^ header[4 + i]);
|
||||
if (raw.length === 4) return { ip: raw.join('.'), port };
|
||||
const groups: string[] = [];
|
||||
for (let i = 0; i < 16; i += 2) groups.push(((raw[i] << 8) | raw[i + 1]).toString(16));
|
||||
return { ip: groups.join(':'), port };
|
||||
}
|
||||
|
||||
interface Stun { type: number; header: Uint8Array; attrs: { type: number; value: Uint8Array; offset: number }[]; raw: Uint8Array }
|
||||
|
||||
export function parse(msg: Uint8Array): Stun | null {
|
||||
if (msg.length < 20) return null;
|
||||
const view = new DataView(msg.buffer, msg.byteOffset, msg.byteLength);
|
||||
if (view.getUint32(4) !== COOKIE || view.getUint16(2) !== msg.length - 20) return null;
|
||||
const attrs: Stun['attrs'] = [];
|
||||
for (let at = 20; at + 4 <= msg.length;) {
|
||||
const type = view.getUint16(at), length = view.getUint16(at + 2);
|
||||
if (at + 4 + length > msg.length) return null;
|
||||
attrs.push({ type, value: msg.subarray(at + 4, at + 4 + length), offset: at });
|
||||
at += 4 + ((length + 3) & ~3);
|
||||
}
|
||||
return { type: view.getUint16(0), header: msg.subarray(0, 20), attrs, raw: msg };
|
||||
}
|
||||
|
||||
// Builds a message; with a key it is signed (MESSAGE-INTEGRITY), which browsers check on answers
|
||||
export function build(type: number, header: Uint8Array, attrs: [number, Uint8Array][], key?: Buffer): Uint8Array {
|
||||
const body = attrs.reduce((n, [, v]) => n + 4 + ((v.length + 3) & ~3), 0);
|
||||
const out = new Uint8Array(20 + body + (key ? 24 : 0));
|
||||
const view = new DataView(out.buffer);
|
||||
view.setUint16(0, type);
|
||||
out.set(header.subarray(4, 20), 4);
|
||||
let at = 20;
|
||||
for (const [t, v] of attrs) {
|
||||
view.setUint16(at, t);
|
||||
view.setUint16(at + 2, v.length);
|
||||
out.set(v, at + 4);
|
||||
at += 4 + ((v.length + 3) & ~3);
|
||||
}
|
||||
// The length field counts the integrity attribute while it is being computed
|
||||
view.setUint16(2, out.length - 20);
|
||||
if (key) {
|
||||
view.setUint16(at, A.MESSAGE_INTEGRITY);
|
||||
view.setUint16(at + 2, 20);
|
||||
out.set(createHmac('sha1', key).update(out.subarray(0, at)).digest(), at + 4);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const text = (s: string) => new TextEncoder().encode(s);
|
||||
const u32 = (n: number) => { const b = new Uint8Array(4); new DataView(b.buffer).setUint32(0, n); return b; };
|
||||
const errorCode = (code: number, reason: string) => Uint8Array.from([0, 0, Math.floor(code / 100), code % 100, ...text(reason)]);
|
||||
|
||||
// Binding answer on its own, for the STUN-only case and the tests
|
||||
export function stunResponse(msg: Uint8Array, address: string, port: number): Uint8Array | null {
|
||||
const m = parse(msg);
|
||||
if (!m || m.type !== BINDING) return null;
|
||||
const mapped = xorAddress(address, port, m.header);
|
||||
return mapped && build(BINDING | 0x100, m.header, [[A.XOR_MAPPED_ADDRESS, mapped]]);
|
||||
}
|
||||
|
||||
// The password that goes with a username, as handed out by the API and checked here
|
||||
export function turnCredential(secret: string, username: string): string {
|
||||
return createHmac('sha1', secret).update(username).digest('base64');
|
||||
}
|
||||
|
||||
interface Client { key: string; ip: string; port: number; send(bytes: Uint8Array): void; local: string | null }
|
||||
interface Allocation {
|
||||
client: Client;
|
||||
authKey: Buffer;
|
||||
relay: dgram.Socket;
|
||||
relayIp: string;
|
||||
permissions: Map<string, number>;
|
||||
channels: Map<number, { ip: string; port: number; expires: number }>;
|
||||
byPeer: Map<string, number>;
|
||||
expires: number;
|
||||
}
|
||||
|
||||
export async function startTurn(opts: TurnOptions, secret: string): Promise<{ port: number; close(): void }> {
|
||||
const allocations = new Map<string, Allocation>();
|
||||
const now = () => Math.floor(Date.now() / 1000);
|
||||
|
||||
const nonce = () => { const t = String(now()); return `${t}-${createHmac('sha1', secret).update(`nonce:${t}`).digest('hex').slice(0, 24)}`; };
|
||||
const nonceValid = (n: string) => {
|
||||
const [t, mac] = n.split('-');
|
||||
return !!mac && now() - Number(t) < NONCE_AGE && mac === createHmac('sha1', secret).update(`nonce:${t}`).digest('hex').slice(0, 24);
|
||||
};
|
||||
|
||||
function free(a: Allocation): void {
|
||||
if (allocations.get(a.client.key) === a) allocations.delete(a.client.key);
|
||||
try { a.relay.close(); } catch { /* already closed */ }
|
||||
}
|
||||
const sweep = setInterval(() => { for (const a of allocations.values()) if (a.expires < now()) free(a); }, 15000);
|
||||
sweep.unref();
|
||||
|
||||
// The address peers must send to. Behind a 1:1 NAT (many cloud hosts) it has to be configured.
|
||||
function relayAddress(family: 4 | 6, local: string | null): string | null {
|
||||
if (opts.publicIp && (net.isIPv4(opts.publicIp) ? 4 : 6) === family) return opts.publicIp;
|
||||
if (local && (net.isIPv4(local) ? 4 : 6) === family && !/^(0\.0\.0\.0|::)$/.test(local)) return local;
|
||||
const want = family === 4 ? 'IPv4' : 'IPv6';
|
||||
const found = Object.values(os.networkInterfaces()).flat().filter(i => i && i.family === want && !i.address.startsWith('fe80'));
|
||||
// A public address first; Docker bridges and the like come before loopback only
|
||||
return (found.find(i => !isPrivateAddress(i!.address)) ?? found.find(i => !i!.internal) ?? found[0])?.address ?? null;
|
||||
}
|
||||
|
||||
async function bindRelay(family: 4 | 6): Promise<dgram.Socket | null> {
|
||||
const span = opts.maxPort - opts.minPort + 1;
|
||||
const start = Math.floor(Math.random() * span);
|
||||
for (let i = 0; i < Math.min(span, 64); i++) {
|
||||
const socket = dgram.createSocket({ type: family === 4 ? 'udp4' : 'udp6', ipv6Only: family === 6 });
|
||||
const ok = await new Promise<boolean>(resolve => {
|
||||
socket.once('error', () => resolve(false));
|
||||
socket.bind(opts.minPort + ((start + i) % span), () => resolve(true));
|
||||
});
|
||||
if (ok) { socket.removeAllListeners('error'); socket.on('error', () => {}); return socket; }
|
||||
try { socket.close(); } catch { /* never opened */ }
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Checks the credentials of a request; answers with the right error and returns null otherwise
|
||||
function authenticate(m: Stun, client: Client): Buffer | null {
|
||||
const get = (t: number) => m.attrs.find(a => a.type === t);
|
||||
const fail = (code: number, reason: string) => {
|
||||
client.send(build(m.type | 0x110, m.header, [[A.ERROR_CODE, errorCode(code, reason)], [A.REALM, text(REALM)], [A.NONCE, text(nonce())]]));
|
||||
return null;
|
||||
};
|
||||
const integrity = get(A.MESSAGE_INTEGRITY);
|
||||
// The first request always comes without credentials; that is how a client learns the realm
|
||||
if (!integrity) return fail(401, 'Unauthorized');
|
||||
opts.log?.(`relay: ${client.key} sent credentials`);
|
||||
const username = get(A.USERNAME), n = get(A.NONCE);
|
||||
if (!username || !n || integrity.value.length !== 20) return fail(400, 'Bad Request');
|
||||
if (!nonceValid(new TextDecoder().decode(n.value))) return fail(438, 'Stale Nonce');
|
||||
const user = new TextDecoder().decode(username.value);
|
||||
// The username is the time the credential runs out
|
||||
if (!(Number(user.split(':')[0]) > now())) return fail(401, 'Unauthorized');
|
||||
const key = createHash('md5').update(`${user}:${REALM}:${turnCredential(secret, user)}`).digest();
|
||||
const signed = Uint8Array.from(m.raw.subarray(0, integrity.offset));
|
||||
new DataView(signed.buffer).setUint16(2, integrity.offset + 24 - 20);
|
||||
const expected = createHmac('sha1', key).update(signed).digest();
|
||||
if (!timingSafeEqual(expected, integrity.value)) return fail(401, 'Unauthorized');
|
||||
return key;
|
||||
}
|
||||
|
||||
// Two clients of this relay reach each other at its announced address. Behind a 1:1 NAT that
|
||||
// address is not on this machine and the router will not turn the packet around, so such
|
||||
// traffic stays on loopback and is presented as coming from the announced address.
|
||||
const loopback = (ip: string) => ip === '127.0.0.1' || ip === '::1';
|
||||
function toPeer(a: Allocation, data: Uint8Array, ip: string, port: number): void {
|
||||
a.relay.send(data, port, ip === a.relayIp ? (net.isIPv4(ip) ? '127.0.0.1' : '::1') : ip);
|
||||
}
|
||||
|
||||
function fromPeer(a: Allocation, data: Buffer, peerIp: string, peerPort: number): void {
|
||||
const seen = v4(peerIp);
|
||||
const ip = loopback(seen) && !loopback(a.relayIp) ? a.relayIp : seen;
|
||||
if ((a.permissions.get(ip) ?? 0) < now()) return;
|
||||
const channel = a.byPeer.get(`${ip}:${peerPort}`);
|
||||
if (channel != null && (a.channels.get(channel)?.expires ?? 0) >= now()) {
|
||||
const out = new Uint8Array(4 + data.length);
|
||||
new DataView(out.buffer).setUint16(0, channel);
|
||||
new DataView(out.buffer).setUint16(2, data.length);
|
||||
out.set(data, 4);
|
||||
return a.client.send(out);
|
||||
}
|
||||
const header = new Uint8Array(20);
|
||||
new DataView(header.buffer).setUint32(4, COOKIE);
|
||||
header.set(randomBytes(12), 8);
|
||||
const peer = xorAddress(ip, peerPort, header);
|
||||
if (peer) a.client.send(build(DATA, header, [[A.XOR_PEER_ADDRESS, peer], [A.DATA, data]]));
|
||||
}
|
||||
|
||||
async function handle(msg: Uint8Array, client: Client): Promise<void> {
|
||||
// ChannelData: channel number, length, payload
|
||||
if (msg.length >= 4 && (msg[0] & 0xc0) === 0x40) {
|
||||
const a = allocations.get(client.key);
|
||||
const view = new DataView(msg.buffer, msg.byteOffset, msg.byteLength);
|
||||
const bound = a?.channels.get(view.getUint16(0));
|
||||
const length = view.getUint16(2);
|
||||
if (a && bound && bound.expires >= now() && 4 + length <= msg.length) toPeer(a, msg.subarray(4, 4 + length), bound.ip, bound.port);
|
||||
return;
|
||||
}
|
||||
const m = parse(msg);
|
||||
if (!m) return;
|
||||
if (m.type === BINDING) {
|
||||
const mapped = xorAddress(client.ip, client.port, m.header);
|
||||
if (mapped) client.send(build(BINDING | 0x100, m.header, [[A.XOR_MAPPED_ADDRESS, mapped]]));
|
||||
return;
|
||||
}
|
||||
if (!opts.relay) return;
|
||||
const get = (t: number) => m.attrs.find(a => a.type === t);
|
||||
const existing = allocations.get(client.key);
|
||||
|
||||
if (m.type === SEND) {
|
||||
const peer = get(A.XOR_PEER_ADDRESS), data = get(A.DATA);
|
||||
const to = peer && unxorAddress(peer.value, m.header);
|
||||
if (existing && to && data && (existing.permissions.get(to.ip) ?? 0) >= now()) toPeer(existing, data.value, to.ip, to.port);
|
||||
return;
|
||||
}
|
||||
if (![ALLOCATE, REFRESH, CREATE_PERMISSION, CHANNEL_BIND].includes(m.type)) return;
|
||||
const key = authenticate(m, client);
|
||||
if (!key) return;
|
||||
const ok = (attrs: [number, Uint8Array][] = []) => client.send(build(m.type | 0x100, m.header, attrs, key));
|
||||
const error = (code: number, reason: string) => {
|
||||
opts.log?.(`relay: ${client.key} refused, ${code} ${reason}`);
|
||||
client.send(build(m.type | 0x110, m.header, [[A.ERROR_CODE, errorCode(code, reason)]], key));
|
||||
};
|
||||
|
||||
if (m.type === ALLOCATE) {
|
||||
if (existing) return error(437, 'Allocation Mismatch');
|
||||
if (get(A.REQUESTED_TRANSPORT)?.value[0] !== 17) return error(442, 'Unsupported Transport Protocol');
|
||||
const family: 4 | 6 = get(A.REQUESTED_ADDRESS_FAMILY)?.value[0] === 2 ? 6 : 4;
|
||||
const mine = [...allocations.values()].filter(a => a.client.ip === client.ip).length;
|
||||
if (allocations.size >= opts.maxAllocations || mine >= opts.maxPerAddress) return error(486, 'Allocation Quota Reached');
|
||||
const relayIp = relayAddress(family, client.local);
|
||||
const relay = relayIp ? await bindRelay(family) : null;
|
||||
if (!relay || !relayIp) return error(508, 'Insufficient Capacity');
|
||||
// Another request for the same client may have won while the port was being opened
|
||||
if (allocations.has(client.key)) { relay.close(); return error(437, 'Allocation Mismatch'); }
|
||||
const a: Allocation = { client, authKey: key, relay, relayIp, permissions: new Map(), channels: new Map(), byPeer: new Map(), expires: now() + LIFETIME };
|
||||
allocations.set(client.key, a);
|
||||
relay.on('message', (data, from) => fromPeer(a, data, from.address, from.port));
|
||||
opts.log?.(`relay: ${client.key} got ${relayIp}:${relay.address().port} (${allocations.size} in use)`);
|
||||
return ok([
|
||||
[A.XOR_RELAYED_ADDRESS, xorAddress(relayIp, relay.address().port, m.header)!],
|
||||
[A.LIFETIME, u32(LIFETIME)],
|
||||
[A.XOR_MAPPED_ADDRESS, xorAddress(client.ip, client.port, m.header)!]
|
||||
]);
|
||||
}
|
||||
if (!existing) return error(437, 'Allocation Mismatch');
|
||||
|
||||
if (m.type === REFRESH) {
|
||||
const wanted = get(A.LIFETIME);
|
||||
const seconds = wanted ? Math.min(new DataView(wanted.value.buffer, wanted.value.byteOffset).getUint32(0), LIFETIME) : LIFETIME;
|
||||
if (seconds === 0) free(existing);
|
||||
else existing.expires = now() + seconds;
|
||||
return ok([[A.LIFETIME, u32(seconds)]]);
|
||||
}
|
||||
const peers = m.attrs.filter(a => a.type === A.XOR_PEER_ADDRESS).map(a => unxorAddress(a.value, m.header));
|
||||
if (!peers.length || peers.some(p => !p)) return error(400, 'Bad Request');
|
||||
// Other clients of this relay are reached at its own address
|
||||
if (peers.some(p => p!.ip !== existing.relayIp && !opts.peerAllowed(p!.ip))) return error(403, 'Forbidden');
|
||||
|
||||
if (m.type === CREATE_PERMISSION) {
|
||||
for (const p of peers) existing.permissions.set(p!.ip, now() + PERMISSION);
|
||||
return ok();
|
||||
}
|
||||
// ChannelBind
|
||||
const number = get(A.CHANNEL_NUMBER);
|
||||
const channel = number ? new DataView(number.value.buffer, number.value.byteOffset).getUint16(0) : 0;
|
||||
const peer = peers[0]!;
|
||||
const peerKey = `${peer.ip}:${peer.port}`;
|
||||
if (channel < 0x4000 || channel > 0x7ffe) return error(400, 'Bad Request');
|
||||
const boundTo = existing.channels.get(channel), boundAs = existing.byPeer.get(peerKey);
|
||||
if ((boundTo && `${boundTo.ip}:${boundTo.port}` !== peerKey) || (boundAs != null && boundAs !== channel)) return error(400, 'Bad Request');
|
||||
existing.channels.set(channel, { ...peer, expires: now() + CHANNEL });
|
||||
existing.byPeer.set(peerKey, channel);
|
||||
existing.permissions.set(peer.ip, now() + PERMISSION);
|
||||
return ok();
|
||||
}
|
||||
|
||||
// UDP: one message per packet
|
||||
const udp = dgram.createSocket(net.isIPv4(opts.bind) ? 'udp4' : 'udp6');
|
||||
udp.on('message', (msg, from) => {
|
||||
const ip = v4(from.address);
|
||||
handle(msg, { key: `udp:${ip}:${from.port}`, ip, port: from.port, local: null, send: bytes => udp.send(bytes, from.port, from.address) }).catch(() => {});
|
||||
});
|
||||
const port = await new Promise<number>((resolve, reject) => {
|
||||
udp.once('error', reject);
|
||||
udp.bind(opts.port, opts.bind, () => { udp.removeAllListeners('error'); udp.on('error', () => {}); resolve(udp.address().port); });
|
||||
});
|
||||
|
||||
// TCP, for browsers and networks that allow no UDP: the same messages, one after another, padded to 4 bytes
|
||||
const tcp = net.createServer(socket => {
|
||||
const ip = v4(socket.remoteAddress ?? ''), remotePort = socket.remotePort ?? 0;
|
||||
const client: Client = {
|
||||
key: `tcp:${ip}:${remotePort}`, ip, port: remotePort, local: socket.localAddress ? v4(socket.localAddress) : null,
|
||||
send: bytes => {
|
||||
if (socket.destroyed) return;
|
||||
// A slow reader must not grow our memory; it loses packets instead, like on UDP
|
||||
if (socket.writableLength > 4 * 1024 * 1024) return;
|
||||
const padded = (bytes.length + 3) & ~3;
|
||||
socket.write(padded === bytes.length ? bytes : Buffer.concat([bytes, Buffer.alloc(padded - bytes.length)]));
|
||||
}
|
||||
};
|
||||
let pending: Buffer = Buffer.alloc(0);
|
||||
socket.setNoDelay(true);
|
||||
socket.setTimeout(LIFETIME * 1000, () => socket.destroy());
|
||||
socket.on('data', (chunk: Buffer) => {
|
||||
pending = pending.length ? Buffer.concat([pending, chunk]) : chunk;
|
||||
while (pending.length >= 4) {
|
||||
const kind = pending[0] & 0xc0;
|
||||
if (kind !== 0x00 && kind !== 0x40) return socket.destroy();
|
||||
const length = (kind === 0x00 ? 20 : 4) + pending.readUInt16BE(2);
|
||||
const framed = (length + 3) & ~3;
|
||||
if (length > 65536) return socket.destroy();
|
||||
if (pending.length < framed) break;
|
||||
handle(Uint8Array.from(pending.subarray(0, length)), client).catch(() => {});
|
||||
pending = pending.subarray(framed);
|
||||
}
|
||||
});
|
||||
socket.on('error', () => {});
|
||||
socket.on('close', () => { const a = allocations.get(client.key); if (a) free(a); });
|
||||
});
|
||||
if (opts.relay) {
|
||||
await new Promise<void>(resolve => {
|
||||
// Relaying over UDP still works without the TCP port
|
||||
tcp.once('error', () => resolve());
|
||||
tcp.listen({ port, host: opts.bind }, () => { tcp.removeAllListeners('error'); tcp.on('error', () => {}); resolve(); });
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
port,
|
||||
close() {
|
||||
clearInterval(sweep);
|
||||
for (const a of [...allocations.values()]) free(a);
|
||||
try { udp.close(); } catch { /* already closed */ }
|
||||
tcp.close();
|
||||
}
|
||||
};
|
||||
}
|
||||
+22
-5
@@ -15,6 +15,9 @@
|
||||
import ChannelDialog from './ui/ChannelDialog.svelte';
|
||||
import PublicServers from './ui/PublicServers.svelte';
|
||||
import ServerInfoDialog from './ui/ServerInfoDialog.svelte';
|
||||
import ImageViewer from './ui/ImageViewer.svelte';
|
||||
import ShareDialog from './ui/ShareDialog.svelte';
|
||||
import { share } from './lib/share.svelte.ts';
|
||||
import ResizeHandle from './ui/ResizeHandle.svelte';
|
||||
import { identities } from './lib/identities.svelte.ts';
|
||||
import { session } from './lib/session.svelte.ts';
|
||||
@@ -26,8 +29,8 @@
|
||||
const layout = $derived(width >= 1100 ? 'wide' : width >= 760 ? 'medium' : 'narrow');
|
||||
|
||||
// Arrangement (Appearance setting): side is the default, classic puts the channel list right
|
||||
// of the chat, stacked above it. Phones always use drawers.
|
||||
const arrangement = $derived(layout === 'narrow' ? 'side' : store.settings.arrangement ?? 'side');
|
||||
// of the chat, stacked above it. Narrow windows use drawers, except stacked, which works at any width.
|
||||
const arrangement = $derived(store.settings.arrangement === 'stacked' ? 'stacked' : layout === 'narrow' ? 'side' : store.settings.arrangement ?? 'side');
|
||||
// The member panel sits inline only in the side arrangement on wide windows, otherwise it overlays
|
||||
const panelInline = $derived(layout === 'wide' && arrangement === 'side');
|
||||
let treeHeight = $state(Math.max(120, Number((() => { try { return localStorage.getItem('mumh5.treeHeight'); } catch { return null; } })()) || 280));
|
||||
@@ -61,20 +64,32 @@
|
||||
else ui.panelOpen = false;
|
||||
}
|
||||
|
||||
// Images in rendered text (messages, descriptions, the quick preview) and image embeds open
|
||||
// large on click. Not in editors, link preview cards or the YouTube card.
|
||||
function imageClick(e: MouseEvent) {
|
||||
if (e.button !== 0 || !(e.target instanceof HTMLImageElement)) return;
|
||||
const img = e.target;
|
||||
if (!img.closest('.html, .embed') || img.closest('[contenteditable], .rich, .card, .yt')) return;
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
ui.image = img.currentSrc || img.src;
|
||||
}
|
||||
|
||||
const closeOverlays = () => {
|
||||
navOpen = false;
|
||||
if (!panelInline) ui.panelOpen = false;
|
||||
};
|
||||
</script>
|
||||
|
||||
<svelte:window bind:innerWidth={width} />
|
||||
<svelte:window bind:innerWidth={width} onclickcapture={imageClick} />
|
||||
|
||||
{#snippet rail()}
|
||||
<ServerRail onadd={() => (editing = { server: null })} onedit={s => (editing = { server: s })} onbrowse={() => (ui.publicServers = true)}
|
||||
onsettings={() => (ui.settingsOpen = true)} />
|
||||
{/snippet}
|
||||
{#snippet chat()}
|
||||
<Chat showMenu={layout === 'narrow'} showStage={!panelInline && arrangement === 'side'} onmenu={() => (navOpen = true)} onpanel={togglePanel} />
|
||||
<Chat showMenu={layout === 'narrow' && arrangement === 'side'} showStage={!panelInline && arrangement === 'side'} videoOnly={arrangement === 'stacked'}
|
||||
onmenu={() => (navOpen = true)} onpanel={togglePanel} />
|
||||
{/snippet}
|
||||
|
||||
<div class="app {layout} {arrangement}">
|
||||
@@ -129,7 +144,9 @@
|
||||
{#if ui.channelDialog && session.status === 'connected'}
|
||||
{#key ui.channelDialog}<ChannelDialog {...ui.channelDialog} onclose={() => (ui.channelDialog = null)} />{/key}
|
||||
{/if}
|
||||
{#if ui.prompt}<PromptDialog {...ui.prompt} onclose={() => (ui.prompt = null)} />{/if}
|
||||
{#if share.dialog}<ShareDialog onclose={() => (share.dialog = null)} />{/if}
|
||||
{#if ui.prompt}<PromptDialog {...ui.prompt} onclose={() => { const p = ui.prompt; ui.prompt = null; p?.oncancel?.(); }} />{/if}
|
||||
<ImageViewer />
|
||||
<ContextMenu />
|
||||
|
||||
<style>
|
||||
|
||||
@@ -73,6 +73,10 @@ label { display: block; font-size: 12px; font-weight: 600; text-transform: upper
|
||||
.icon-btn:hover { background: var(--bg-hover); color: var(--text); }
|
||||
.icon-btn.on { color: var(--danger); }
|
||||
|
||||
/* Images in rendered text open large on click (App.svelte) */
|
||||
.html img, .embed img { cursor: zoom-in; }
|
||||
.card img { cursor: inherit; }
|
||||
|
||||
::-webkit-scrollbar { width: 10px; height: 10px; }
|
||||
::-webkit-scrollbar-thumb { background: var(--bg-3); border: 2px solid transparent; background-clip: padding-box; }
|
||||
::-webkit-scrollbar-thumb:hover { background-color: var(--bg-active); }
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
// Screen-share signalling between mumh5 clients, carried as Mumble plugin data so it works on
|
||||
// any server and other clients ignore it. Murmur drops plugin data over 1000 bytes, so session
|
||||
// descriptions are compressed and split into chunks.
|
||||
export const SHARE_DATA_ID = 'mumh5.share';
|
||||
|
||||
export const SHARE = {
|
||||
on: 1, // I am sharing my screen
|
||||
off: 2, // I stopped
|
||||
who: 3, // I just got here: who is sharing?
|
||||
watch: 4, // let me watch
|
||||
leave: 5, // I stopped watching
|
||||
offer: 6, // session description from the sharer
|
||||
answer: 7 // session description from the viewer
|
||||
} as const;
|
||||
export type ShareType = typeof SHARE[keyof typeof SHARE];
|
||||
|
||||
const CHUNK = 900;
|
||||
const MAX_CHUNKS = 32;
|
||||
const MAX_TEXT = 256 * 1024;
|
||||
let nextId = 0;
|
||||
|
||||
async function deflate(bytes: Uint8Array): Promise<Uint8Array> {
|
||||
const out = new Blob([bytes as Uint8Array<ArrayBuffer>]).stream().pipeThrough(new CompressionStream('deflate-raw'));
|
||||
return new Uint8Array(await new Response(out).arrayBuffer());
|
||||
}
|
||||
|
||||
// Stops reading once the output passes the cap, so a tiny message cannot expand without bound
|
||||
async function inflate(bytes: Uint8Array): Promise<Uint8Array | null> {
|
||||
const reader = new Blob([bytes as Uint8Array<ArrayBuffer>]).stream().pipeThrough(new DecompressionStream('deflate-raw')).getReader();
|
||||
const parts: Uint8Array[] = [];
|
||||
let size = 0;
|
||||
try {
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
size += value.length;
|
||||
if (size > MAX_TEXT) { await reader.cancel(); return null; }
|
||||
parts.push(value);
|
||||
}
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const all = new Uint8Array(size);
|
||||
let at = 0;
|
||||
for (const p of parts) { all.set(p, at); at += p.length; }
|
||||
return all;
|
||||
}
|
||||
|
||||
// One packet for a bare signal, several for a signal with text: [type, id, index, total, ...data]
|
||||
export async function encodeShare(type: ShareType, text?: string): Promise<Uint8Array[]> {
|
||||
if (text == null) return [new Uint8Array([type])];
|
||||
const data = await deflate(new TextEncoder().encode(text));
|
||||
const total = Math.max(1, Math.ceil(data.length / CHUNK));
|
||||
if (total > MAX_CHUNKS) throw new Error('Signal too large');
|
||||
const id = nextId = (nextId + 1) & 255;
|
||||
return Array.from({ length: total }, (_, i) => {
|
||||
const part = data.subarray(i * CHUNK, (i + 1) * CHUNK);
|
||||
const packet = new Uint8Array(4 + part.length);
|
||||
packet.set([type, id, i, total]);
|
||||
packet.set(part, 4);
|
||||
return packet;
|
||||
});
|
||||
}
|
||||
|
||||
export interface ShareSignal { type: ShareType; text?: string }
|
||||
|
||||
// Puts chunked signals back together, per sender
|
||||
export class ShareAssembler {
|
||||
private pending = new Map<string, { parts: (Uint8Array | undefined)[]; got: number; at: number }>();
|
||||
|
||||
async push(sender: number, packet: Uint8Array, now = Date.now()): Promise<ShareSignal | null> {
|
||||
const type = packet[0] as ShareType;
|
||||
if (!Object.values(SHARE).includes(type)) return null;
|
||||
if (packet.length === 1) return { type };
|
||||
if (packet.length < 4) return null;
|
||||
const [, id, index, total] = packet;
|
||||
if (!total || total > MAX_CHUNKS || index >= total) return null;
|
||||
// Abandoned transfers must not pile up
|
||||
for (const [k, v] of this.pending) if (now - v.at > 15000) this.pending.delete(k);
|
||||
if (this.pending.size > 64) this.pending.clear();
|
||||
const key = `${sender}:${type}:${id}`;
|
||||
let entry = this.pending.get(key);
|
||||
if (!entry || entry.parts.length !== total) {
|
||||
entry = { parts: new Array(total), got: 0, at: now };
|
||||
this.pending.set(key, entry);
|
||||
}
|
||||
if (!entry.parts[index]) entry.got++;
|
||||
entry.parts[index] = packet.slice(4);
|
||||
if (entry.got < total) return null;
|
||||
this.pending.delete(key);
|
||||
const joined = new Uint8Array(entry.parts.reduce((n, p) => n + p!.length, 0));
|
||||
let at = 0;
|
||||
for (const p of entry.parts) { joined.set(p!, at); at += p!.length; }
|
||||
const plain = await inflate(joined);
|
||||
return plain ? { type, text: new TextDecoder().decode(plain) } : null;
|
||||
}
|
||||
}
|
||||
+10
-2
@@ -1,4 +1,5 @@
|
||||
import { session } from './session.svelte.ts';
|
||||
import { session, sessions } from './session.svelte.ts';
|
||||
import { share } from './share.svelte.ts';
|
||||
import { ui } from './ui.svelte.ts';
|
||||
import { store } from './settings.svelte.ts';
|
||||
import { voice, userKey } from './audio/voice.svelte.ts';
|
||||
@@ -38,7 +39,8 @@ export function channelMenu(c: Channel): MenuItem[] {
|
||||
return [
|
||||
{ header: true, label: c.name || 'Root' },
|
||||
{ label: here ? 'You are here' : 'Join channel', action: () => session.joinChannel(c.id), disabled: here || !c.canEnter },
|
||||
{ label: 'Open chat', action: () => { session.showChannel(c.id, 'chat'); ui.panelOpen = true; } },
|
||||
// Opens the channel's chat in the side panel, ready to type, without joining it
|
||||
{ label: 'Send message to channel', action: () => { session.showChannel(c.id, 'chat'); ui.panelOpen = true; } },
|
||||
{ label: 'Show description', action: () => { session.showChannel(c.id, 'info'); ui.panelOpen = true; } },
|
||||
{ separator: true, label: '' },
|
||||
{ label: session.isExpanded(c.id) ? 'Collapse' : 'Expand', action: () => session.toggleExpanded(c.id), disabled: c.id === 0 },
|
||||
@@ -67,6 +69,12 @@ export function userMenu(u: User): MenuItem[] {
|
||||
);
|
||||
if (u.userId == null) items.push({ label: 'Register on this server', action: () => session.register(u.session) });
|
||||
} else {
|
||||
const sharing = session.sharing[u.session];
|
||||
if (sharing === 'camera') {
|
||||
// Cameras show by themselves; this is the per-person switch
|
||||
const hidden = share.isHidden(sessions.active, u.session) || !share.watchOf(u.session);
|
||||
items.push({ label: hidden ? 'Show camera' : 'Hide camera', action: () => share.hideCamera(sessions.active, u.session, !hidden) });
|
||||
} else if (sharing) items.push({ label: `Watch ${sharing}`, action: () => share.watch(sessions.active, u.session) });
|
||||
items.push(
|
||||
{ label: 'Send message', action: () => session.selectView(`dm:${u.session}`) },
|
||||
{ label: 'Mute for me', checked: !!voice.settings.localMutes[userKey(u)], action: () => voice.setLocalMute(u, !voice.settings.localMutes[userKey(u)]) },
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
import wasmUrl from '@evan/opus/wasm/opus.wasm?url';
|
||||
import { instantiateOpus, type Opus } from './opus-wasm.ts';
|
||||
|
||||
// Fetched once, and only in browsers that need it
|
||||
let loading: Promise<Opus> | null = null;
|
||||
export function loadOpus(): Promise<Opus> {
|
||||
return loading ??= fetch(wasmUrl).then(r => r.arrayBuffer()).then(instantiateOpus).catch(e => { loading = null; throw e; });
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
// Opus in WebAssembly (libopus, from the @evan/opus package), for browsers without WebCodecs
|
||||
// audio, such as Firefox on Android. Only loaded there; everywhere else the browser's own
|
||||
// codec is used. The two classes offer the small part of the WebCodecs interface the voice
|
||||
// engine needs. No DOM or bundler imports here, so the tests can run it in Node.
|
||||
|
||||
interface Exports {
|
||||
memory: WebAssembly.Memory;
|
||||
malloc(size: number): number;
|
||||
free(ptr: number): void;
|
||||
opus_encoder_get_size(channels: number): number;
|
||||
opus_encoder_init(ptr: number, rate: number, channels: number, application: number): number;
|
||||
opus_encoder_ctl_set(ptr: number, request: number, value: number): number;
|
||||
opus_encode(ptr: number, pcm: number, frameSize: number, out: number, max: number): number;
|
||||
opus_decoder_get_size(channels: number): number;
|
||||
opus_decoder_init(ptr: number, rate: number, channels: number): number;
|
||||
opus_decode(ptr: number, data: number, length: number, pcm: number, frameSize: number, fec: number): number;
|
||||
}
|
||||
|
||||
export interface Opus { wasm: Exports }
|
||||
|
||||
const RATE = 48000;
|
||||
const MAX_PACKET = 4000;
|
||||
// The longest Opus frame is 120 ms
|
||||
const MAX_SAMPLES = 5760;
|
||||
const APPLICATION_VOIP = 2048, SET_BITRATE = 4002, SET_COMPLEXITY = 4010, SET_INBAND_FEC = 4012, SET_PACKET_LOSS = 4014, SET_SIGNAL = 4024, SIGNAL_VOICE = 3001;
|
||||
|
||||
// bytes: the contents of opus.wasm (fetched by opus-load.ts in the app, read from disk in tests)
|
||||
export async function instantiateOpus(bytes: BufferSource): Promise<Opus> {
|
||||
const imports = {
|
||||
wasi_snapshot_preview1: { fd_seek() { return 0; }, fd_write() { return 0; }, fd_close() { return 0; }, proc_exit() {} },
|
||||
// Memory is read through fresh views every call, so growth needs no bookkeeping
|
||||
env: { emscripten_notify_memory_growth() {} }
|
||||
};
|
||||
const { instance } = await WebAssembly.instantiate(bytes, imports);
|
||||
return { wasm: instance.exports as unknown as Exports };
|
||||
}
|
||||
|
||||
// Takes the engine's 10 ms frames and gives out one packet per frame duration
|
||||
export class WasmEncoder {
|
||||
state: 'configured' | 'closed' = 'configured';
|
||||
private wasm: Exports;
|
||||
private ptr: number;
|
||||
private pcm: number;
|
||||
private out: number;
|
||||
private samples: number;
|
||||
private filled = 0;
|
||||
private output: (packet: Uint8Array) => void;
|
||||
|
||||
constructor(opus: Opus, bitrate: number, frameMs: number, output: (packet: Uint8Array) => void) {
|
||||
const w = this.wasm = opus.wasm;
|
||||
this.output = output;
|
||||
this.samples = RATE * frameMs / 1000;
|
||||
this.ptr = w.malloc(w.opus_encoder_get_size(1));
|
||||
this.pcm = w.malloc(this.samples * 2);
|
||||
this.out = w.malloc(MAX_PACKET);
|
||||
if (w.opus_encoder_init(this.ptr, RATE, 1, APPLICATION_VOIP) < 0) { this.close(); throw new Error('Could not start the Opus encoder'); }
|
||||
w.opus_encoder_ctl_set(this.ptr, SET_BITRATE, Math.round(bitrate));
|
||||
// A notch below the maximum: phones are the ones without a codec of their own
|
||||
w.opus_encoder_ctl_set(this.ptr, SET_COMPLEXITY, 8);
|
||||
w.opus_encoder_ctl_set(this.ptr, SET_INBAND_FEC, 1);
|
||||
w.opus_encoder_ctl_set(this.ptr, SET_PACKET_LOSS, 10);
|
||||
w.opus_encoder_ctl_set(this.ptr, SET_SIGNAL, SIGNAL_VOICE);
|
||||
}
|
||||
|
||||
push(frame: Float32Array): void {
|
||||
if (this.state !== 'configured') return;
|
||||
let view = new Int16Array(this.wasm.memory.buffer, this.pcm, this.samples);
|
||||
for (let i = 0; i < frame.length; i++) {
|
||||
const v = Math.max(-1, Math.min(1, frame[i]));
|
||||
view[this.filled++] = v < 0 ? v * 0x8000 : v * 0x7fff;
|
||||
if (this.filled < this.samples) continue;
|
||||
this.filled = 0;
|
||||
const length = this.wasm.opus_encode(this.ptr, this.pcm, this.samples, this.out, MAX_PACKET);
|
||||
if (length > 0) this.output(new Uint8Array(this.wasm.memory.buffer, this.out, length).slice());
|
||||
// Encoding may have grown the memory, which detaches the view
|
||||
view = new Int16Array(this.wasm.memory.buffer, this.pcm, this.samples);
|
||||
}
|
||||
}
|
||||
|
||||
// An unfinished frame at the end of a transmission is dropped
|
||||
flush(): Promise<void> {
|
||||
this.filled = 0;
|
||||
return Promise.resolve();
|
||||
}
|
||||
|
||||
close(): void {
|
||||
if (this.state === 'closed') return;
|
||||
this.state = 'closed';
|
||||
for (const p of [this.ptr, this.pcm, this.out]) if (p) this.wasm.free(p);
|
||||
}
|
||||
}
|
||||
|
||||
export class WasmDecoder {
|
||||
private wasm: Exports;
|
||||
private ptr: number;
|
||||
private data: number;
|
||||
private pcm: number;
|
||||
private closed = false;
|
||||
|
||||
constructor(opus: Opus) {
|
||||
const w = this.wasm = opus.wasm;
|
||||
this.ptr = w.malloc(w.opus_decoder_get_size(1));
|
||||
this.data = w.malloc(MAX_PACKET);
|
||||
this.pcm = w.malloc(MAX_SAMPLES * 2);
|
||||
if (w.opus_decoder_init(this.ptr, RATE, 1) < 0) { this.close(); throw new Error('Could not start the Opus decoder'); }
|
||||
}
|
||||
|
||||
// Mono samples for the playback worklet, or null for a packet that does not decode
|
||||
decode(packet: Uint8Array): Float32Array<ArrayBuffer> | null {
|
||||
// An empty packet would make libopus invent audio (loss concealment)
|
||||
if (this.closed || !packet.length || packet.length > MAX_PACKET) return null;
|
||||
new Uint8Array(this.wasm.memory.buffer, this.data, packet.length).set(packet);
|
||||
const n = this.wasm.opus_decode(this.ptr, this.data, packet.length, this.pcm, MAX_SAMPLES, 0);
|
||||
if (n <= 0) return null;
|
||||
const samples = new Int16Array(this.wasm.memory.buffer, this.pcm, n);
|
||||
const out = new Float32Array(n);
|
||||
for (let i = 0; i < n; i++) out[i] = samples[i] / 0x8000;
|
||||
return out;
|
||||
}
|
||||
|
||||
close(): void {
|
||||
if (this.closed) return;
|
||||
this.closed = true;
|
||||
for (const p of [this.ptr, this.data, this.pcm]) if (p) this.wasm.free(p);
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,10 @@ export const SOUND_EVENTS = [
|
||||
{ id: 'userBanned', group: 'Your channel', name: 'Someone in your channel was banned', ts: 'CLIENT_WAS_BANNED_CURRENT_CHANNEL', keys: ['banned_server_currentchannel'] },
|
||||
{ id: 'recordingStarted', group: 'Your channel', name: 'Someone started recording', ts: 'CLIENT_RECORDING_START', keys: ['recording_started'] },
|
||||
{ id: 'recordingStopped', group: 'Your channel', name: 'Someone stopped recording', ts: 'CLIENT_RECORDING_STOP', keys: ['recording_stopped'] },
|
||||
{ id: 'shareStarted', group: 'Screen sharing', name: 'A screen share started (yours or in your channel)', ts: 'MUMH5_SHARE_STARTED', keys: ['share_started', 'stream_started'] },
|
||||
{ id: 'shareStopped', group: 'Screen sharing', name: 'A screen share stopped', ts: 'MUMH5_SHARE_STOPPED', keys: ['share_stopped', 'stream_stopped'] },
|
||||
{ id: 'viewerJoined', group: 'Screen sharing', name: 'Someone started watching your screen', ts: 'MUMH5_VIEWER_JOINED', keys: ['viewer_joined'] },
|
||||
{ id: 'viewerLeft', group: 'Screen sharing', name: 'Someone stopped watching your screen', ts: 'MUMH5_VIEWER_LEFT', keys: ['viewer_left'] },
|
||||
{ id: 'message', group: 'Chat', name: 'Channel message', ts: 'CHAT_RECEIVED_MESSAGE_CHANNEL', keys: ['chat_message_inbound', 'message'] },
|
||||
{ id: 'privateMessage', group: 'Chat', name: 'Direct message', ts: 'CHAT_RECEIVED_MESSAGE_CLIENT', keys: ['private', 'direct', 'pm'] },
|
||||
{ id: 'messageSent', group: 'Chat', name: 'You sent a message', ts: 'CHAT_SENT_MESSAGE_CHANNEL', keys: ['chat_message_outbound', 'sent'], off: true },
|
||||
@@ -79,6 +83,10 @@ const TONES: Record<SoundEvent, [number, number, number][]> = {
|
||||
userBanned: [[294, 0, 0.12], [220, 0.13, 0.16]],
|
||||
recordingStarted: [[1175, 0, 0.08], [1175, 0.12, 0.08]],
|
||||
recordingStopped: [[880, 0, 0.12]],
|
||||
shareStarted: [[523, 0, 0.07], [784, 0.08, 0.07], [1047, 0.16, 0.12]],
|
||||
shareStopped: [[1047, 0, 0.07], [784, 0.08, 0.07], [523, 0.16, 0.12]],
|
||||
viewerJoined: [[880, 0, 0.05], [1109, 0.06, 0.08]],
|
||||
viewerLeft: [[1109, 0, 0.05], [880, 0.06, 0.08]],
|
||||
message: [[1047, 0, 0.06]],
|
||||
privateMessage: [[1047, 0, 0.05], [1319, 0.07, 0.07]],
|
||||
messageSent: [[1319, 0, 0.04]],
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { WasmEncoder, WasmDecoder, type Opus } from './opus-wasm.ts';
|
||||
import { loadOpus } from './opus-load.ts';
|
||||
import captureUrl from './capture.worklet.ts?worker&url';
|
||||
import playbackUrl from './playback.worklet.ts?worker&url';
|
||||
import { encodeVoice, decodeVoice } from '../../core/voice-packet.ts';
|
||||
@@ -77,8 +79,10 @@ class VoiceEngine {
|
||||
private outputGainNode: GainNode | null = null;
|
||||
private capture: AudioWorkletNode | null = null;
|
||||
private playback: AudioWorkletNode | null = null;
|
||||
private encoder: AudioEncoder | null = null;
|
||||
private decoders = new Map<number, { decoder: AudioDecoder; lastFrame: number; timer: ReturnType<typeof setTimeout> | null }>();
|
||||
// The browser's Opus codec (WebCodecs), or our own in WebAssembly where the browser has none
|
||||
private opus: Opus | null = null;
|
||||
private encoder: AudioEncoder | WasmEncoder | null = null;
|
||||
private decoders = new Map<number, { decoder: AudioDecoder | WasmDecoder; lastFrame: number; timer: ReturnType<typeof setTimeout> | null }>();
|
||||
private client: MumbleClient | null = null;
|
||||
private sessionGains = new Map<number, number>();
|
||||
private unsubscribe: (() => void)[] = [];
|
||||
@@ -139,8 +143,18 @@ class VoiceEngine {
|
||||
if (this.ctx) return;
|
||||
this.error = '';
|
||||
try {
|
||||
if (typeof AudioEncoder === 'undefined' || !(await AudioEncoder.isConfigSupported(this.encoderConfig())).supported) {
|
||||
throw new Error('This system cannot encode Opus audio');
|
||||
const builtIn = typeof AudioEncoder !== 'undefined' && typeof AudioDecoder !== 'undefined' &&
|
||||
(await AudioEncoder.isConfigSupported(this.encoderConfig()).catch(() => ({ supported: false }))).supported;
|
||||
// mumh5.forceWasmOpus in localStorage: use the built-in codec regardless, for testing it
|
||||
let forced = false;
|
||||
try { forced = localStorage.getItem('mumh5.forceWasmOpus') === '1'; } catch { /* storage unavailable */ }
|
||||
if (!builtIn || forced) {
|
||||
// Firefox on Android, for one, has no audio codec for pages; bring our own
|
||||
try {
|
||||
this.opus = await loadOpus();
|
||||
} catch (e) {
|
||||
throw new Error(`This browser has no Opus audio codec, and the built-in one could not be loaded (${(e as Error).message})`);
|
||||
}
|
||||
}
|
||||
const ctx = new AudioContext({ sampleRate: SAMPLE_RATE, latencyHint: 'interactive' });
|
||||
this.ctx = ctx;
|
||||
@@ -281,11 +295,16 @@ class VoiceEngine {
|
||||
this.effectiveBitrate = this.allowedBitrate();
|
||||
if (!this.ctx) return;
|
||||
try { this.encoder?.close(); } catch { /* closed */ }
|
||||
this.encoder = new AudioEncoder({
|
||||
if (this.opus) {
|
||||
this.encoder = new WasmEncoder(this.opus, this.effectiveBitrate, this.settings.frameMs, packet => this.onPacket(packet));
|
||||
return;
|
||||
}
|
||||
const encoder = new AudioEncoder({
|
||||
output: chunk => this.onEncoded(chunk),
|
||||
error: e => { this.error = `Encoder: ${e.message}`; }
|
||||
});
|
||||
this.encoder.configure(this.encoderConfig());
|
||||
encoder.configure(this.encoderConfig());
|
||||
this.encoder = encoder;
|
||||
}
|
||||
|
||||
private canTalk(): boolean {
|
||||
@@ -320,18 +339,24 @@ class VoiceEngine {
|
||||
if (!wanted && this.open) { this.endTransmission(); return; }
|
||||
if (!this.open) return;
|
||||
|
||||
const encoder = this.encoder!;
|
||||
if (encoder instanceof WasmEncoder) return encoder.push(frame);
|
||||
const data = new AudioData({
|
||||
format: 'f32-planar', sampleRate: SAMPLE_RATE, numberOfFrames: FRAME, numberOfChannels: 1,
|
||||
timestamp: this.sampleTime, data: frame
|
||||
});
|
||||
this.sampleTime += 10000;
|
||||
this.encoder!.encode(data);
|
||||
encoder.encode(data);
|
||||
data.close();
|
||||
}
|
||||
|
||||
private onEncoded(chunk: EncodedAudioChunk): void {
|
||||
const opus = new Uint8Array(chunk.byteLength);
|
||||
chunk.copyTo(opus);
|
||||
this.onPacket(opus);
|
||||
}
|
||||
|
||||
private onPacket(opus: Uint8Array): void {
|
||||
if (this.stopping) { this.held.push(opus); return; }
|
||||
this.sendPacket(opus, false);
|
||||
}
|
||||
@@ -389,11 +414,15 @@ class VoiceEngine {
|
||||
let d = this.decoders.get(p.session);
|
||||
if (!d) {
|
||||
const session = p.session;
|
||||
const decoder = new AudioDecoder({
|
||||
output: data => this.onDecoded(session, data),
|
||||
error: () => this.dropSession(session)
|
||||
});
|
||||
decoder.configure({ codec: 'opus', sampleRate: SAMPLE_RATE, numberOfChannels: 1 });
|
||||
let decoder: AudioDecoder | WasmDecoder;
|
||||
if (this.opus) decoder = new WasmDecoder(this.opus);
|
||||
else {
|
||||
decoder = new AudioDecoder({
|
||||
output: data => this.onDecoded(session, data),
|
||||
error: () => this.dropSession(session)
|
||||
});
|
||||
decoder.configure({ codec: 'opus', sampleRate: SAMPLE_RATE, numberOfChannels: 1 });
|
||||
}
|
||||
d = { decoder, lastFrame: -1, timer: null };
|
||||
this.decoders.set(session, d);
|
||||
if (user) this.setSessionGain(session, this.settings.userVolumes[userKey(user)] ?? 1);
|
||||
@@ -402,7 +431,13 @@ class VoiceEngine {
|
||||
d.lastFrame = p.frame;
|
||||
|
||||
if (p.opus.length) {
|
||||
d.decoder.decode(new EncodedAudioChunk({ type: 'key', timestamp: p.frame * 10000, data: p.opus }));
|
||||
if (d.decoder instanceof WasmDecoder) {
|
||||
const pcm = d.decoder.decode(p.opus);
|
||||
if (pcm) {
|
||||
this.stats.decoded++;
|
||||
this.playback?.port.postMessage({ session: p.session, pcm }, [pcm.buffer]);
|
||||
}
|
||||
} else d.decoder.decode(new EncodedAudioChunk({ type: 'key', timestamp: p.frame * 10000, data: p.opus }));
|
||||
}
|
||||
if (!this.talking[p.session]) this.talking[p.session] = true;
|
||||
if (d.timer) clearTimeout(d.timer);
|
||||
|
||||
@@ -6,6 +6,11 @@ export const ICON_PATHS: Record<string, string> = {
|
||||
'headphones-off': 'M3 14v-2a9 9 0 0 1 18 0v2M21 14v7h-4v-7zM3 14v7h4v-7zM3 3l18 18',
|
||||
settings: 'M4 21v-7M4 10V3M12 21v-9M12 8V3M20 21v-5M20 12V3M1 14h6M9 8h6M17 16h6',
|
||||
plus: 'M12 5v14M5 12h14',
|
||||
switch: 'M4 8h13l-3-3M20 16H7l3 3',
|
||||
camera: 'M2 6h14v12H2zM16 10l6-3v10l-6-3',
|
||||
screen: 'M3 4h18v12H3zM8 20h8M12 16v4',
|
||||
popout: 'M14 4h6v6M20 4l-9 9M18 14v6H4V6h6',
|
||||
expand: 'M4 9V4h5M20 9V4h-5M4 15v5h5M20 15v5h-5',
|
||||
filter: 'M3 4h18l-7 8v7l-4 2v-9z',
|
||||
info: 'M12 2a10 10 0 1 0 0 20 10 10 0 0 0 0-20zM12 11v6M12 7h.01',
|
||||
'arrow-right': 'M5 12h14M13 6l6 6-6 6',
|
||||
|
||||
@@ -29,6 +29,17 @@ interface NativeApi extends PlatformApi {
|
||||
pingServer(host: string, port: number): Promise<PingResult | null>;
|
||||
onContextMenu(fn: (params: ContextParams) => void): void;
|
||||
editAction(action: string, arg?: unknown): void;
|
||||
// Screens and windows that can be shared; pick one before calling getDisplayMedia
|
||||
// chosen: the system's own picker already asked (Wayland), so `sources` is the one answer
|
||||
// Thumbnails are slower to get and can fail; ask for the names first
|
||||
screenSources(thumbnails: boolean): Promise<{ sources: ScreenSource[]; chosen: boolean }>;
|
||||
pickScreenSource(id: string): void;
|
||||
// portal: the system shows its own source picker (Wayland). audio: how other programs' sound can be captured.
|
||||
shareInfo(): Promise<{ portal: boolean; audio: 'pipewire' | 'loopback' | 'none'; micLabel: string }>;
|
||||
shareAudioApps(): Promise<string[]>;
|
||||
// Linux: route one program, or everything but this app, into a virtual microphone named micLabel
|
||||
shareAudioStart(target: { kind: 'all' } | { kind: 'app'; name: string }): Promise<void>;
|
||||
shareAudioStop(): Promise<void>;
|
||||
open(connId: string, host: string, port: number, identityId?: string): Promise<void>;
|
||||
send(connId: string, bytes: Uint8Array): void;
|
||||
close(connId: string): void;
|
||||
@@ -53,6 +64,7 @@ export interface ContextParams {
|
||||
editFlags: { canUndo: boolean; canRedo: boolean; canCut: boolean; canCopy: boolean; canPaste: boolean; canSelectAll: boolean };
|
||||
}
|
||||
|
||||
export interface ScreenSource { id: string; name: string; thumbnail: string }
|
||||
export interface PublicServer { name: string; host: string; port: number; url: string; country: string; countryCode: string; continent: string; ca: boolean }
|
||||
export interface PingResult { ping: number; users: number; maxUsers: number; bandwidth: number; version: string }
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { createCodec } from '../core/proto.ts';
|
||||
import { MumbleClient, describeDenial, type Channel, type User, type TextMessage } from '../core/client.ts';
|
||||
import { native, openTransport } from './native.ts';
|
||||
import { share, type ShareKind } from './share.svelte.ts';
|
||||
import type { ServerCertInfo } from '../core/transport.ts';
|
||||
import { store, type SavedServer } from './settings.svelte.ts';
|
||||
import { renderIncoming, textToHtml, escapeHtml, formatBytes, type Embed } from './html.ts';
|
||||
@@ -60,6 +61,8 @@ export class Session {
|
||||
uploads = $state<UploadJob[]>([]);
|
||||
certPrompt = $state<CertPrompt | null>(null);
|
||||
rtt = $state(0);
|
||||
// Who in our channel is sharing their screen, by session (mumh5 clients only)
|
||||
sharing = $state<Record<number, ShareKind>>({});
|
||||
// Why the last connection ended, shown with a Reconnect button (null after a normal disconnect)
|
||||
disconnectInfo = $state<{ kind: 'kicked' | 'banned' | 'lost' | 'rejected'; by: string; reason: string } | null>(null);
|
||||
private removedBy: { kind: 'kicked' | 'banned'; by: string; reason: string } | null = null;
|
||||
@@ -288,6 +291,8 @@ export class Session {
|
||||
const client = this.client;
|
||||
this.client = null;
|
||||
client?.disconnect();
|
||||
share.closed(this);
|
||||
this.sharing = {};
|
||||
sessions.onClosed(this);
|
||||
this.status = 'idle';
|
||||
this.tick++;
|
||||
@@ -312,8 +317,10 @@ export class Session {
|
||||
// What we may do per channel, for menus (not rate limited by the server)
|
||||
for (const id of client.channels.keys()) client.requestPermissions(id);
|
||||
if (client.welcomeText) this.system(this.view, client.welcomeText, true);
|
||||
share.entered(this);
|
||||
bump();
|
||||
});
|
||||
client.on('pluginData', (sender, dataId, data) => { if (live()) share.receive(this, sender, dataId, data); });
|
||||
client.on('channel', ch => {
|
||||
if (live() && client.synced && !client.permissions.has(ch.id)) client.requestPermissions(ch.id);
|
||||
bump();
|
||||
@@ -337,6 +344,10 @@ export class Session {
|
||||
const self = client.self;
|
||||
const before = this.lastChannel.get(user.session);
|
||||
this.lastChannel.set(user.session, user.channelId);
|
||||
if (client.synced && self && (isNew || changed.includes('channelId'))) {
|
||||
if (user.session === self.session) share.entered(this);
|
||||
else share.userMoved(this, user);
|
||||
}
|
||||
if (client.synced && self && this.isVoice()) this.userSound(user.session === self.session, user, changed, actor, isNew, before, self.channelId);
|
||||
if (client.synced && self) {
|
||||
if (isNew) {
|
||||
@@ -366,6 +377,7 @@ export class Session {
|
||||
this.removedBy = { kind: ban ? 'banned' : 'kicked', by, reason };
|
||||
}
|
||||
const self = client.self;
|
||||
share.userMoved(this, user, true);
|
||||
if (self && user.session !== self.session && this.isVoice() && user.channelId === self.channelId) {
|
||||
const byOther = actor != null && actor !== user.session;
|
||||
sounds.play(ban ? 'userBanned' : byOther ? 'userKicked' : 'userDisconnected');
|
||||
@@ -675,6 +687,12 @@ export class Session {
|
||||
|
||||
// ─── Voice state and moderation ─────────────────────────────────────────────
|
||||
|
||||
setSharing(session: number, kind: ShareKind | false): void {
|
||||
if (kind) this.sharing[session] = kind;
|
||||
else delete this.sharing[session];
|
||||
}
|
||||
clearSharing(): void { this.sharing = {}; }
|
||||
|
||||
// The choice is remembered, so it also works while disconnected and carries to the next connect
|
||||
setSelfMute(v: boolean) {
|
||||
sessions.mute = v;
|
||||
|
||||
@@ -24,6 +24,12 @@ export interface Settings {
|
||||
theme: ThemeId;
|
||||
publicListUrl: string; // where the public server list comes from
|
||||
arrangement: Arrangement;
|
||||
shareQuality: 'low' | 'medium' | 'high'; // bitrate and frame rate of our own stream
|
||||
autoCameras: boolean; // show other people's cameras as they come on
|
||||
shareVolume: number; // volume of streams we watch, 0 to 1
|
||||
relayServer: string; // a mumh5 web proxy whose STUN and relay to use for screen sharing
|
||||
stunServer: string; // helps screen sharing connect through routers; empty contacts nobody
|
||||
shareNoticeSeen: boolean; // the note that screen sharing reveals IP addresses was confirmed
|
||||
hints: boolean; // tooltips (title attributes) when resting the pointer on buttons
|
||||
}
|
||||
|
||||
@@ -52,6 +58,12 @@ const defaults: Settings = {
|
||||
theme: 'dark',
|
||||
publicListUrl: 'https://publist.mumble.info/v1/list?version=1.5.0',
|
||||
arrangement: 'side',
|
||||
shareQuality: 'medium',
|
||||
autoCameras: true,
|
||||
shareVolume: 1,
|
||||
relayServer: '',
|
||||
stunServer: '',
|
||||
shareNoticeSeen: false,
|
||||
hints: false
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,474 @@
|
||||
// Screen sharing between mumh5 clients in the same channel. Mumble has no video, so the
|
||||
// picture goes directly from the sharer to each viewer over WebRTC; only the setup messages
|
||||
// travel through the Mumble server (as plugin data, see core/share-signal.ts). The server has
|
||||
// checked the sender's certificate, so a stream is known to come from that user.
|
||||
import { SHARE, SHARE_DATA_ID, encodeShare, ShareAssembler, type ShareType } from '../core/share-signal.ts';
|
||||
import type { User } from '../core/client.ts';
|
||||
import type { Session } from './session.svelte.ts';
|
||||
import { desktop, isWeb } from './native.ts';
|
||||
import { proxyIce, relayIce } from './web.svelte.ts';
|
||||
import { store } from './settings.svelte.ts';
|
||||
import { ui } from './ui.svelte.ts';
|
||||
import { sounds } from './audio/sounds.svelte.ts';
|
||||
|
||||
// Every viewer is a separate upload from the sharer
|
||||
const MAX_VIEWERS = 8;
|
||||
// Cameras watched at once; each is a download and a decoder
|
||||
const MAX_CAMERAS = 12;
|
||||
// A face needs far less than a screen full of text
|
||||
const CAMERA_BITRATE = 1_200_000;
|
||||
export const QUALITY = {
|
||||
low: { label: 'Low (1 Mbit/s, 30 fps)', bitrate: 1_000_000, fps: 30 },
|
||||
medium: { label: 'Medium (4 Mbit/s, 30 fps)', bitrate: 4_000_000, fps: 30 },
|
||||
high: { label: 'High (8 Mbit/s, 60 fps)', bitrate: 8_000_000, fps: 60 }
|
||||
} as const;
|
||||
|
||||
// Opus is negotiated as mono speech by default; shared sound is stereo and often music
|
||||
const stereo = (sdp: string) => sdp.replace(/a=fmtp:(\d+) ([^\r\n]*useinbandfec=1[^\r\n]*)/g,
|
||||
(line, pt, rest) => rest.includes('stereo=1') ? line : `a=fmtp:${pt} ${rest};stereo=1;sprop-stereo=1;maxaveragebitrate=192000`);
|
||||
|
||||
// What a stream shows. Sent along with the announcement, so others see the right icon.
|
||||
export type ShareKind = 'screen' | 'camera';
|
||||
|
||||
export interface Watching {
|
||||
host: Session;
|
||||
session: number;
|
||||
kind: ShareKind;
|
||||
stream: MediaStream | null;
|
||||
state: 'connecting' | 'live' | 'failed';
|
||||
}
|
||||
|
||||
class ScreenShare {
|
||||
// The server we are sharing on, and what we send
|
||||
host = $state.raw<Session | null>(null);
|
||||
stream = $state.raw<MediaStream | null>(null);
|
||||
kind = $state<ShareKind>('screen');
|
||||
// Show our own camera mirrored to ourselves, as a mirror would. Not a camera pointing away
|
||||
// from us (a phone's back camera), and never what the others get.
|
||||
mirror = $state(false);
|
||||
// How many cameras this device has; with more than one there is something to switch to
|
||||
cameras = $state(0);
|
||||
switching = $state(false);
|
||||
viewers = $state(0);
|
||||
// Streams we receive: one screen at a time, and any number of cameras next to it
|
||||
watches = $state.raw<Watching[]>([]);
|
||||
// People whose camera we chose not to see, by certificate (or name); until the app restarts
|
||||
hidden = $state<Record<string, boolean>>({});
|
||||
error = $state('');
|
||||
// The server the start dialog is open for
|
||||
dialog = $state.raw<Session | null>(null);
|
||||
private usesVirtualMic = false;
|
||||
|
||||
private peers = new Map<number, RTCPeerConnection>();
|
||||
private watchPcs = new Map<number, RTCPeerConnection>();
|
||||
private watchTimers = new Map<number, ReturnType<typeof setTimeout>>();
|
||||
private assemblers = new WeakMap<Session, ShareAssembler>();
|
||||
// Viewers whose connection came up, so leaving is only announced for those
|
||||
private live = new Set<number>();
|
||||
// When we entered the channel: streams already running there are not announced with a sound
|
||||
private enteredAt = new WeakMap<Session, number>();
|
||||
|
||||
// Plugin data needs a 1.4 server; older ones would not pass the signals on
|
||||
supported(s: Session): boolean {
|
||||
return !!s.client && s.client.serverVersionNum >= 0x010400 && typeof RTCPeerConnection !== 'undefined';
|
||||
}
|
||||
|
||||
private async iceServers(): Promise<RTCIceServer[]> {
|
||||
const stun = store.settings.stunServer.trim();
|
||||
const chosen: RTCIceServer[] = stun ? [{ urls: /^stuns?:/.test(stun) ? stun : `stun:${stun}` }] : [];
|
||||
// A mumh5 proxy adds its STUN and its relay, which takes over by itself when no direct
|
||||
// connection comes up (mobile and company networks, browsers that forbid direct UDP). The
|
||||
// browser build has the proxy it is served from; the desktop app the one set in Settings.
|
||||
const relay = store.settings.relayServer.trim();
|
||||
return [...chosen, ...(relay ? await relayIce(relay) : isWeb ? await proxyIce() : [])];
|
||||
}
|
||||
|
||||
// For testing the relay: localStorage mumh5.iceDebug = "relay" uses only relayed routes,
|
||||
// "relay-tcp" only the relay reached over TCP (what a browser without direct UDP is left with)
|
||||
private async rtcConfig(): Promise<RTCConfiguration> {
|
||||
const iceServers = await this.iceServers();
|
||||
let debug = '';
|
||||
try { debug = localStorage.getItem('mumh5.iceDebug') ?? ''; } catch { /* storage unavailable */ }
|
||||
if (!debug.startsWith('relay')) return { iceServers };
|
||||
const tcpOnly = (urls: string | string[]) => [urls].flat().filter(u => !u.startsWith('turn') || u.includes('transport=tcp'));
|
||||
return { iceTransportPolicy: 'relay', iceServers: debug === 'relay-tcp' ? iceServers.map(s => ({ ...s, urls: tcpOnly(s.urls) })) : iceServers };
|
||||
}
|
||||
|
||||
private others(s: Session): number[] {
|
||||
const self = s.client?.self;
|
||||
if (!self) return [];
|
||||
return [...s.client!.users.values()].filter(u => u.session !== self.session && u.channelId === self.channelId).map(u => u.session);
|
||||
}
|
||||
|
||||
private inMyChannel(s: Session, session: number): boolean {
|
||||
const self = s.client?.self;
|
||||
return !!self && s.client!.users.get(session)?.channelId === self.channelId;
|
||||
}
|
||||
|
||||
private async send(s: Session, to: number[], type: ShareType, text?: string): Promise<void> {
|
||||
if (!to.length) return;
|
||||
for (const packet of await encodeShare(type, text)) s.client?.sendPluginData(to, SHARE_DATA_ID, packet);
|
||||
}
|
||||
|
||||
// Sharer and viewer connect directly, so each learns the other's address; say so once
|
||||
private notice(): Promise<boolean> {
|
||||
if (store.settings.shareNoticeSeen) return Promise.resolve(true);
|
||||
return new Promise(resolve => {
|
||||
let answered = false;
|
||||
ui.prompt = {
|
||||
title: 'Screen sharing', label: '', confirm: 'Continue', input: 'none',
|
||||
text: 'Screen sharing connects you directly to other participants in this channel. You see each other\'s IP address. Only mumh5 users in your channel can watch.',
|
||||
onsubmit: () => { answered = true; store.settings.shareNoticeSeen = true; store.saveSettings(); resolve(true); },
|
||||
oncancel: () => { if (!answered) resolve(false); }
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
// Opens the dialog where the source, the sound and the quality are chosen
|
||||
async start(s: Session): Promise<void> {
|
||||
if (this.stream || this.dialog || !this.supported(s) || !(await this.notice())) return;
|
||||
this.error = '';
|
||||
this.dialog = s;
|
||||
}
|
||||
|
||||
// The dialog hands over what it captured
|
||||
go(s: Session, stream: MediaStream, usesVirtualMic: boolean, kind: ShareKind = 'screen'): void {
|
||||
this.dialog = null;
|
||||
// Disconnected while the dialog was open
|
||||
if (!s.client || this.stream) {
|
||||
stream.getTracks().forEach(t => t.stop());
|
||||
if (usesVirtualMic) desktop?.shareAudioStop();
|
||||
return;
|
||||
}
|
||||
for (const t of stream.getVideoTracks()) {
|
||||
// Text on a screen must stay sharp; a face may blur a little but should move smoothly
|
||||
t.contentHint = kind === 'camera' ? 'motion' : 'detail';
|
||||
// The system's "stop sharing" control
|
||||
t.addEventListener('ended', () => this.stop());
|
||||
}
|
||||
this.usesVirtualMic = usesVirtualMic;
|
||||
this.kind = kind;
|
||||
this.mirror = kind === 'camera' && stream.getVideoTracks()[0]?.getSettings().facingMode !== 'environment';
|
||||
this.cameras = 0;
|
||||
if (kind === 'camera') navigator.mediaDevices.enumerateDevices().then(d => { this.cameras = d.filter(x => x.kind === 'videoinput').length; }).catch(() => {});
|
||||
this.host = s;
|
||||
this.stream = stream;
|
||||
this.send(s, this.others(s), SHARE.on, kind);
|
||||
s.setSharing(s.client.session!, kind);
|
||||
sounds.play('shareStarted');
|
||||
}
|
||||
|
||||
// The next camera of this device (front and back on a phone), without interrupting the stream
|
||||
async switchCamera(): Promise<void> {
|
||||
const stream = this.stream;
|
||||
const old = stream?.getVideoTracks()[0];
|
||||
if (!stream || !old || this.kind !== 'camera' || this.switching) return;
|
||||
this.switching = true;
|
||||
this.error = '';
|
||||
const open = (deviceId: string) => navigator.mediaDevices.getUserMedia({
|
||||
video: { deviceId: { exact: deviceId }, width: { ideal: 1280 }, height: { ideal: 720 }, frameRate: { ideal: QUALITY[store.settings.shareQuality].fps } }
|
||||
}).catch(() => navigator.mediaDevices.getUserMedia({ video: { deviceId: { exact: deviceId } } }));
|
||||
try {
|
||||
const ids = (await navigator.mediaDevices.enumerateDevices()).filter(d => d.kind === 'videoinput').map(d => d.deviceId);
|
||||
this.cameras = ids.length;
|
||||
const current = old.getSettings().deviceId ?? '';
|
||||
const next = ids[(ids.indexOf(current) + 1) % ids.length];
|
||||
if (ids.length < 2 || !next || next === current) return;
|
||||
let fresh: MediaStream;
|
||||
try {
|
||||
// Computers can have both open for a moment, which keeps the picture going
|
||||
fresh = await open(next);
|
||||
} catch {
|
||||
// A phone opens one camera at a time, so the old one has to go first
|
||||
old.stop();
|
||||
try {
|
||||
fresh = await open(next);
|
||||
} catch (e) {
|
||||
// Could not get the other one: back to the camera we had
|
||||
fresh = await open(current).catch(() => { throw e; });
|
||||
this.error = `Could not switch the camera: ${(e as Error).message}`;
|
||||
}
|
||||
}
|
||||
// Stopped sharing in the meantime
|
||||
if (this.stream !== stream) return fresh.getTracks().forEach(t => t.stop());
|
||||
const track = fresh.getVideoTracks()[0];
|
||||
track.contentHint = 'motion';
|
||||
track.addEventListener('ended', () => { if (this.stream?.getVideoTracks()[0] === track) this.stop(); });
|
||||
// Viewers keep their connection; only the picture changes
|
||||
for (const pc of this.peers.values()) {
|
||||
for (const sender of pc.getSenders()) {
|
||||
if (sender.track !== old) continue;
|
||||
await sender.replaceTrack(track).catch(e => { this.error = `A viewer did not get the new camera: ${(e as Error).message}`; });
|
||||
}
|
||||
}
|
||||
old.stop();
|
||||
this.stream = new MediaStream([track, ...stream.getAudioTracks()]);
|
||||
this.mirror = track.getSettings().facingMode !== 'environment';
|
||||
} catch (e) {
|
||||
this.error = `The camera stopped: ${(e as Error).message}`;
|
||||
this.stop();
|
||||
} finally {
|
||||
this.switching = false;
|
||||
}
|
||||
}
|
||||
|
||||
private release(): void {
|
||||
this.dropPeers();
|
||||
this.stream?.getTracks().forEach(t => t.stop());
|
||||
this.stream = null;
|
||||
this.host = null;
|
||||
if (this.usesVirtualMic) desktop?.shareAudioStop();
|
||||
this.usesVirtualMic = false;
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
const s = this.host;
|
||||
if (this.stream) sounds.play('shareStopped');
|
||||
if (s?.client) {
|
||||
this.send(s, this.others(s), SHARE.off);
|
||||
if (s.client.session != null) s.setSharing(s.client.session, false);
|
||||
}
|
||||
this.release();
|
||||
}
|
||||
|
||||
private dropPeers(): void {
|
||||
for (const pc of this.peers.values()) pc.close();
|
||||
this.peers.clear();
|
||||
this.live.clear();
|
||||
this.viewers = 0;
|
||||
}
|
||||
|
||||
private dropPeer(session: number): void {
|
||||
this.peers.get(session)?.close();
|
||||
this.peers.delete(session);
|
||||
if (this.live.delete(session)) sounds.play('viewerLeft');
|
||||
this.viewers = this.peers.size;
|
||||
}
|
||||
|
||||
// The screen being watched, if any (there is at most one)
|
||||
get watching(): Watching | null {
|
||||
return this.watches.find(w => w.kind === 'screen') ?? null;
|
||||
}
|
||||
watchOf(session: number): Watching | undefined {
|
||||
return this.watches.find(w => w.session === session);
|
||||
}
|
||||
|
||||
// Cameras come on by themselves, like in a video call, once the note about IP addresses was
|
||||
// confirmed; each person's can be hidden, and the whole thing switched off
|
||||
get cams(): boolean {
|
||||
return store.settings.autoCameras;
|
||||
}
|
||||
private who(s: Session, session: number): string {
|
||||
const u = s.client?.users.get(session);
|
||||
return u ? u.hash || `name:${u.name}` : '';
|
||||
}
|
||||
isHidden(s: Session, session: number): boolean {
|
||||
return !!this.hidden[this.who(s, session)];
|
||||
}
|
||||
hideCamera(s: Session, session: number, hide: boolean): void {
|
||||
const key = this.who(s, session);
|
||||
if (!key) return;
|
||||
if (hide) { this.hidden[key] = true; this.unwatch(session); }
|
||||
else { delete this.hidden[key]; this.watch(s, session); }
|
||||
}
|
||||
private autoWatch(s: Session, session: number): void {
|
||||
if (this.cams && store.settings.shareNoticeSeen && s.sharing[session] === 'camera' && !this.isHidden(s, session)) this.watch(s, session);
|
||||
}
|
||||
|
||||
async watch(s: Session, session: number): Promise<void> {
|
||||
const kind = s.sharing[session];
|
||||
if (!this.supported(s) || !kind || this.watchOf(session) || !(await this.notice())) return;
|
||||
// Streams from another server, and another screen, make way
|
||||
for (const w of this.watches) if (w.host !== s || (kind === 'screen' && w.kind === 'screen')) this.unwatch(w.session);
|
||||
if (kind === 'camera' && this.watches.filter(w => w.kind === 'camera').length >= MAX_CAMERAS) return;
|
||||
if (!s.sharing[session] || this.watchOf(session)) return;
|
||||
// Asking for it shows it again
|
||||
delete this.hidden[this.who(s, session)];
|
||||
this.watches = [...this.watches, { host: s, session, kind, stream: null, state: 'connecting' }];
|
||||
this.send(s, [session], SHARE.watch);
|
||||
this.watchTimers.set(session, setTimeout(() => this.setWatchState(session, 'failed'), 20000));
|
||||
}
|
||||
|
||||
// One stream, or all of them
|
||||
unwatch(session?: number): void {
|
||||
for (const w of this.watches) {
|
||||
if (session != null && w.session !== session) continue;
|
||||
if (w.host.client) this.send(w.host, [w.session], SHARE.leave);
|
||||
this.endWatch(w.session);
|
||||
}
|
||||
}
|
||||
|
||||
private endWatch(session: number): void {
|
||||
clearTimeout(this.watchTimers.get(session));
|
||||
this.watchTimers.delete(session);
|
||||
this.watchPcs.get(session)?.close();
|
||||
this.watchPcs.delete(session);
|
||||
this.watches = this.watches.filter(w => w.session !== session);
|
||||
}
|
||||
|
||||
private setWatchState(session: number, state: Watching['state'], stream?: MediaStream): void {
|
||||
if (!this.watchOf(session)) return;
|
||||
if (state !== 'connecting') { clearTimeout(this.watchTimers.get(session)); this.watchTimers.delete(session); }
|
||||
this.watches = this.watches.map(w => w.session === session ? { ...w, state, stream: stream ?? w.stream } : w);
|
||||
}
|
||||
|
||||
// All cameras on or off, remembered
|
||||
async setCams(s: Session, on: boolean): Promise<void> {
|
||||
if (on && (!this.supported(s) || !(await this.notice()))) return;
|
||||
store.settings.autoCameras = on;
|
||||
store.saveSettings();
|
||||
if (!on) {
|
||||
for (const w of this.watches) if (w.kind === 'camera') this.unwatch(w.session);
|
||||
return;
|
||||
}
|
||||
for (const session of Object.keys(s.sharing)) if (Number(session) !== s.client?.session) this.autoWatch(s, Number(session));
|
||||
}
|
||||
|
||||
// Candidates travel inside the description, so wait for them instead of sending each one
|
||||
private gathered(pc: RTCPeerConnection): Promise<void> {
|
||||
if (pc.iceGatheringState === 'complete') return Promise.resolve();
|
||||
return new Promise(resolve => {
|
||||
const done = () => { clearTimeout(timer); pc.removeEventListener('icegatheringstatechange', check); resolve(); };
|
||||
const check = () => { if (pc.iceGatheringState === 'complete') done(); };
|
||||
const timer = setTimeout(done, 3000);
|
||||
pc.addEventListener('icegatheringstatechange', check);
|
||||
});
|
||||
}
|
||||
|
||||
private async offerTo(s: Session, viewer: number): Promise<void> {
|
||||
const stream = this.stream;
|
||||
if (!stream || this.host !== s) return;
|
||||
const config = await this.rtcConfig();
|
||||
if (this.stream !== stream || this.host !== s) return;
|
||||
this.dropPeer(viewer);
|
||||
if (this.peers.size >= MAX_VIEWERS) return;
|
||||
const pc = new RTCPeerConnection(config);
|
||||
this.peers.set(viewer, pc);
|
||||
this.viewers = this.peers.size;
|
||||
pc.addEventListener('connectionstatechange', () => {
|
||||
if (this.peers.get(viewer) !== pc) return;
|
||||
if (pc.connectionState === 'connected' && !this.live.has(viewer)) {
|
||||
this.live.add(viewer);
|
||||
sounds.play('viewerJoined');
|
||||
} else if (pc.connectionState === 'failed' || pc.connectionState === 'closed') this.dropPeer(viewer);
|
||||
});
|
||||
for (const track of stream.getTracks()) {
|
||||
const sender = pc.addTrack(track, stream);
|
||||
if (track.kind !== 'video') continue;
|
||||
const params = sender.getParameters();
|
||||
// Keep the picture sharp and let the frame rate drop when the connection is tight
|
||||
params.degradationPreference = this.kind === 'camera' ? 'maintain-framerate' : 'maintain-resolution';
|
||||
const wanted = QUALITY[store.settings.shareQuality].bitrate;
|
||||
for (const enc of params.encodings ?? []) enc.maxBitrate = this.kind === 'camera' ? Math.min(wanted, CAMERA_BITRATE) : wanted;
|
||||
await sender.setParameters(params).catch(() => {});
|
||||
}
|
||||
await pc.setLocalDescription({ type: 'offer', sdp: stereo((await pc.createOffer()).sdp ?? '') });
|
||||
await this.gathered(pc);
|
||||
if (this.peers.get(viewer) === pc && pc.localDescription) await this.send(s, [viewer], SHARE.offer, pc.localDescription.sdp);
|
||||
}
|
||||
|
||||
private async answerTo(s: Session, sharer: number, sdp: string): Promise<void> {
|
||||
const mine = () => { const w = this.watchOf(sharer); return !!w && w.host === s; };
|
||||
if (!mine()) return;
|
||||
const config = await this.rtcConfig();
|
||||
if (!mine()) return;
|
||||
this.watchPcs.get(sharer)?.close();
|
||||
const pc = new RTCPeerConnection(config);
|
||||
this.watchPcs.set(sharer, pc);
|
||||
const current = () => this.watchPcs.get(sharer) === pc;
|
||||
// The track is announced with the description, before any route exists: the stream only
|
||||
// counts as live once the connection is up, otherwise the viewer stares at a black picture
|
||||
pc.addEventListener('track', e => {
|
||||
if (current()) this.setWatchState(sharer, pc.connectionState === 'connected' ? 'live' : 'connecting', e.streams[0]);
|
||||
});
|
||||
pc.addEventListener('connectionstatechange', () => {
|
||||
if (!current()) return;
|
||||
if (pc.connectionState === 'connected') this.setWatchState(sharer, 'live');
|
||||
else if (pc.connectionState === 'failed') this.setWatchState(sharer, 'failed');
|
||||
});
|
||||
try {
|
||||
await pc.setRemoteDescription({ type: 'offer', sdp });
|
||||
await pc.setLocalDescription({ type: 'answer', sdp: stereo((await pc.createAnswer()).sdp ?? '') });
|
||||
await this.gathered(pc);
|
||||
if (current() && pc.localDescription) await this.send(s, [sharer], SHARE.answer, pc.localDescription.sdp);
|
||||
} catch {
|
||||
if (current()) this.setWatchState(sharer, 'failed');
|
||||
}
|
||||
}
|
||||
|
||||
// Raw WebRTC statistics for the overlay: of a stream we watch, or one per viewer of ours
|
||||
async reports(own: boolean, session?: number): Promise<{ viewer: number | null; report: RTCStatsReport }[]> {
|
||||
if (!own) {
|
||||
const pc = session != null ? this.watchPcs.get(session) : undefined;
|
||||
return pc ? [{ viewer: null, report: await pc.getStats() }] : [];
|
||||
}
|
||||
return Promise.all([...this.peers].map(async ([viewer, pc]) => ({ viewer, report: await pc.getStats() })));
|
||||
}
|
||||
|
||||
// ─── Called by the session ────────────────────────────────────────────────
|
||||
|
||||
async receive(s: Session, sender: number, dataId: string, data: Uint8Array): Promise<void> {
|
||||
if (dataId !== SHARE_DATA_ID) return;
|
||||
let assembler = this.assemblers.get(s);
|
||||
if (!assembler) this.assemblers.set(s, assembler = new ShareAssembler());
|
||||
const msg = await assembler.push(sender, data);
|
||||
// Only people in our channel take part
|
||||
if (!msg || !s.client || !this.inMyChannel(s, sender)) return;
|
||||
switch (msg.type) {
|
||||
case SHARE.on:
|
||||
if (!s.sharing[sender] && Date.now() - (this.enteredAt.get(s) ?? 0) > 3000) sounds.play('shareStarted');
|
||||
s.setSharing(sender, msg.text === 'camera' ? 'camera' : 'screen');
|
||||
this.autoWatch(s, sender);
|
||||
break;
|
||||
case SHARE.off:
|
||||
if (s.sharing[sender]) sounds.play('shareStopped');
|
||||
s.setSharing(sender, false);
|
||||
if (this.watchOf(sender)?.host === s) this.endWatch(sender);
|
||||
break;
|
||||
case SHARE.who: if (this.host === s) this.send(s, [sender], SHARE.on, this.kind); break;
|
||||
case SHARE.watch: if (this.host === s) this.offerTo(s, sender).catch(() => this.dropPeer(sender)); break;
|
||||
case SHARE.leave: if (this.host === s) this.dropPeer(sender); break;
|
||||
case SHARE.offer: if (msg.text) this.answerTo(s, sender, msg.text); break;
|
||||
case SHARE.answer:
|
||||
if (this.host === s && msg.text) this.peers.get(sender)?.setRemoteDescription({ type: 'answer', sdp: msg.text }).catch(() => this.dropPeer(sender));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// We connected or moved: ask who is sharing here, and tell the new channel about our stream
|
||||
entered(s: Session): void {
|
||||
if (!this.supported(s)) return;
|
||||
const self = s.client!.session!;
|
||||
this.enteredAt.set(s, Date.now());
|
||||
s.clearSharing();
|
||||
for (const w of this.watches) if (w.host === s) this.unwatch(w.session);
|
||||
if (this.host === s) {
|
||||
this.dropPeers();
|
||||
s.setSharing(self, this.kind);
|
||||
this.send(s, this.others(s), SHARE.on, this.kind);
|
||||
}
|
||||
this.send(s, this.others(s), SHARE.who);
|
||||
}
|
||||
|
||||
// Someone else connected, moved or left
|
||||
userMoved(s: Session, user: User, gone = false): void {
|
||||
if (!s.client?.self || user.session === s.client.session) return;
|
||||
if (!gone && this.inMyChannel(s, user.session)) {
|
||||
if (this.host === s) this.send(s, [user.session], SHARE.on, this.kind);
|
||||
return;
|
||||
}
|
||||
// A sharer who left or disconnected: their stream is over for us
|
||||
if (s.sharing[user.session]) sounds.play('shareStopped');
|
||||
s.setSharing(user.session, false);
|
||||
if (this.host === s) this.dropPeer(user.session);
|
||||
if (this.watchOf(user.session)?.host === s) this.endWatch(user.session);
|
||||
}
|
||||
|
||||
closed(s: Session): void {
|
||||
if (this.host === s) this.release();
|
||||
if (this.dialog === s) this.dialog = null;
|
||||
for (const w of this.watches) if (w.host === s) this.endWatch(w.session);
|
||||
}
|
||||
}
|
||||
|
||||
export const share = new ScreenShare();
|
||||
@@ -9,6 +9,8 @@ export interface PromptRequest {
|
||||
initial?: string;
|
||||
text?: string;
|
||||
onsubmit: (value: string) => void;
|
||||
// Closed without confirming
|
||||
oncancel?: () => void;
|
||||
}
|
||||
|
||||
export interface CertRequest {
|
||||
@@ -28,6 +30,8 @@ class Ui {
|
||||
settingsTab = $state<'voice' | 'sounds' | 'appearance' | 'accessibility' | 'files' | 'identities'>('voice');
|
||||
publicServers = $state(false);
|
||||
serverInfo = $state(false);
|
||||
// Image shown large (clicked in the chat or in a description)
|
||||
image = $state<string | null>(null);
|
||||
// Set by the toolbar to open the own profile with the description editor
|
||||
editDescription = $state(false);
|
||||
// Add-server dialog, optionally prefilled (from the public list)
|
||||
|
||||
+31
-1
@@ -135,14 +135,17 @@ export interface ProxyServer { host: string; port: number; label: string }
|
||||
class ProxyInfo {
|
||||
servers = $state<ProxyServer[]>([]);
|
||||
any = $state(false);
|
||||
// UDP port of the proxy's STUN responder, if it runs one
|
||||
stun = $state<number | null>(null);
|
||||
loaded = $state(false);
|
||||
error = $state('');
|
||||
|
||||
async load(): Promise<void> {
|
||||
try {
|
||||
const c = await call<{ servers: ProxyServer[]; any: boolean }>('config');
|
||||
const c = await call<{ servers: ProxyServer[]; any: boolean; stun?: number | null; turn?: boolean }>('config');
|
||||
this.servers = c.servers;
|
||||
this.any = c.any;
|
||||
this.stun = c.stun ?? null;
|
||||
this.error = '';
|
||||
} catch (e) {
|
||||
this.error = (e as Error).message;
|
||||
@@ -151,3 +154,30 @@ class ProxyInfo {
|
||||
}
|
||||
}
|
||||
export const proxyInfo = new ProxyInfo();
|
||||
|
||||
// STUN and the relay of a mumh5 proxy, for screen sharing. The browser build uses the proxy it
|
||||
// is served from, a host the user already uses; the desktop app one the user entered. The relay
|
||||
// needs credentials, which the proxy hands out for a few hours at a time.
|
||||
const relays = new Map<string, { servers: RTCIceServer[]; until: number }>();
|
||||
export async function relayIce(address: string): Promise<RTCIceServer[]> {
|
||||
let url: URL;
|
||||
try { url = new URL(/^https?:\/\//i.test(address) ? address : `https://${address}`); } catch { return []; }
|
||||
const root = url.href.replace(/\/*$/, '/');
|
||||
const known = relays.get(root);
|
||||
if (known && known.until > Date.now()) return known.servers;
|
||||
try {
|
||||
const res = await fetch(`${root}api/relay`);
|
||||
if (!res.ok) return [];
|
||||
const c = await res.json() as { port: number; username?: string; credential?: string; ttl?: number; urls?: string[] };
|
||||
const servers: RTCIceServer[] = [{ urls: `stun:${url.hostname}:${c.port}` }];
|
||||
// UDP where it is allowed, TCP for networks and browsers that forbid it
|
||||
// The operator may name other ways in, such as a TLS front on port 443 for networks that allow nothing else
|
||||
const direct = [`turn:${url.hostname}:${c.port}?transport=udp`, `turn:${url.hostname}:${c.port}?transport=tcp`];
|
||||
if (c.username) servers.push({ urls: [...direct, ...(c.urls ?? [])], username: c.username, credential: c.credential });
|
||||
relays.set(root, { servers, until: Date.now() + Math.max(60, (c.ttl ?? 3600) - 3000) * 1000 });
|
||||
return servers;
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
export const proxyIce = (): Promise<RTCIceServer[]> => base ? relayIce(base) : Promise.resolve([]);
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
</button>
|
||||
{/each}
|
||||
</div>
|
||||
<p class="help">Narrow windows always use the phone layout.</p>
|
||||
<p class="help">Narrow windows use the phone layout, except Stacked, which stays as it is.</p>
|
||||
|
||||
<style>
|
||||
.gap { margin-top: 22px; }
|
||||
|
||||
@@ -11,6 +11,8 @@
|
||||
import { drag } from '../lib/drag.svelte.ts';
|
||||
import { sounds } from '../lib/audio/sounds.svelte.ts';
|
||||
import { renderIncoming } from '../lib/html.ts';
|
||||
import { share } from '../lib/share.svelte.ts';
|
||||
import { sessions } from '../lib/session.svelte.ts';
|
||||
import type { Channel } from '../core/client.ts';
|
||||
|
||||
let { channel, depth = 0, onnavigate }: { channel: Channel; depth?: number; onnavigate: () => void } = $props();
|
||||
@@ -47,8 +49,20 @@
|
||||
// Short grace period, so the pointer can move from the row into the preview
|
||||
function leaveRow() {
|
||||
clearTimeout(hoverTimer);
|
||||
hoverTimer = setTimeout(() => (preview = null), 250);
|
||||
// An image opened from the preview covers it; the preview stays underneath
|
||||
hoverTimer = setTimeout(() => { if (!ui.image) preview = null; }, 250);
|
||||
}
|
||||
// Back from the image: the preview stays while the pointer is on it, and goes when it is not
|
||||
let popEl = $state<HTMLElement>();
|
||||
let hadImage = false;
|
||||
$effect(() => {
|
||||
const open = !!ui.image;
|
||||
if (hadImage && !open && preview && !popEl?.matches(':hover')) {
|
||||
clearTimeout(hoverTimer);
|
||||
hoverTimer = setTimeout(() => { if (!popEl?.matches(':hover')) preview = null; }, 1500);
|
||||
}
|
||||
hadImage = open;
|
||||
});
|
||||
function showDescription() {
|
||||
clearTimeout(hoverTimer);
|
||||
preview = null;
|
||||
@@ -129,7 +143,7 @@
|
||||
{/if}
|
||||
</div>
|
||||
{#if preview}
|
||||
<div class="desc-pop" role="tooltip" style:left="{preview.x}px" style:top="{preview.y}px"
|
||||
<div class="desc-pop" role="tooltip" bind:this={popEl} style:left="{preview.x}px" style:top="{preview.y}px"
|
||||
onmouseenter={() => clearTimeout(hoverTimer)} onmouseleave={leaveRow}>
|
||||
<strong>{channel.name || 'Root'}</strong>
|
||||
{#if descriptionHtml}<div class="html">{@html descriptionHtml}</div>{:else}<p>Loading description...</p>{/if}
|
||||
@@ -154,6 +168,18 @@
|
||||
<Avatar name={u.name} size={22} speaking={voice.talking[u.session] || (isSelf && voice.transmitting)} />
|
||||
<span class="label" class:talking={voice.talking[u.session] || (isSelf && voice.transmitting)}>{store.displayName(u)}</span>
|
||||
{#if u.userId != null}<span class="reg" title="Registered">R</span>{/if}
|
||||
{#if session.sharing[u.session]}
|
||||
{@const kind = session.sharing[u.session]}
|
||||
{#if isSelf}
|
||||
<span class="state sharing" title="You are sharing your {kind}"><Icon name={kind} size={14} /></span>
|
||||
{:else}
|
||||
<!-- A button cannot sit inside the row's button -->
|
||||
<span class="state sharing" role="button" tabindex="0" title="Sharing their {kind}. Click to watch" aria-label="Watch the {kind} of {store.displayName(u)}"
|
||||
onclick={e => { e.stopPropagation(); clearTimeout(timer); share.watch(sessions.active, u.session); }}
|
||||
ondblclick={e => e.stopPropagation()}
|
||||
onkeydown={e => { if (e.key === 'Enter') { e.stopPropagation(); share.watch(sessions.active, u.session); } }}><Icon name={kind} size={14} /></span>
|
||||
{/if}
|
||||
{/if}
|
||||
{#if u.comment || u.commentHash}<span class="state" title="Has a description"><Icon name="message" size={12} /></span>{/if}
|
||||
{#if u.prioritySpeaker}<span class="state" title="Priority speaker"><Icon name="channel" size={14} /></span>{/if}
|
||||
{#if u.recording}<span class="rec" title="Recording">REC</span>{/if}
|
||||
@@ -207,5 +233,7 @@
|
||||
.reg { font-size: 10px; font-weight: 700; color: var(--accent-hover); border: 1px solid var(--accent); padding: 0 3px; line-height: 14px; }
|
||||
.state { color: var(--text-faint); display: inline-flex; }
|
||||
.state.admin { color: var(--danger); }
|
||||
.state.sharing { color: var(--speaking); padding: 3px; }
|
||||
.state.sharing[role=button]:hover { background: var(--bg-active); }
|
||||
.rec { font-size: 10px; font-weight: 700; color: var(--danger); border: 1px solid var(--danger); padding: 0 3px; }
|
||||
</style>
|
||||
|
||||
+30
-3
@@ -3,11 +3,22 @@
|
||||
import ChatLog from './ChatLog.svelte';
|
||||
import Composer from './Composer.svelte';
|
||||
import VoiceStage from './VoiceStage.svelte';
|
||||
import ResizeHandle from './ResizeHandle.svelte';
|
||||
import { share } from '../lib/share.svelte.ts';
|
||||
import { ui } from '../lib/ui.svelte.ts';
|
||||
import { session } from '../lib/session.svelte.ts';
|
||||
|
||||
// showStage: voice tiles above the chat when the window is too narrow for the member list
|
||||
let { onmenu, onpanel, showMenu, showStage = false }: { onmenu: () => void; onpanel: () => void; showMenu: boolean; showStage?: boolean } = $props();
|
||||
// videoOnly: streams as a plain picture without the tiles (stacked layout)
|
||||
let { onmenu, onpanel, showMenu, showStage = false, videoOnly = false }: { onmenu: () => void; onpanel: () => void; showMenu: boolean; showStage?: boolean; videoOnly?: boolean } = $props();
|
||||
|
||||
// Stacked layout with a stream above the chat: the chat's height can be dragged larger than
|
||||
// its minimum; the picture gets the rest. Unset until dragged, then the picture decides.
|
||||
const LOG_MIN = 124;
|
||||
const here = (s: { server?: { id: string } | null } | null | undefined) => !!s && s.server?.id === session.server?.id;
|
||||
const streamAbove = $derived(videoOnly && session.status === 'connected' && ((!!share.stream && here(share.host)) || share.watches.some(w => here(w.host))));
|
||||
let logHeight = $state(Number((() => { try { return localStorage.getItem('mumh5.stackedChatHeight'); } catch { return null; } })()) || 0);
|
||||
const logFixed = $derived(streamAbove && logHeight >= LOG_MIN);
|
||||
|
||||
const isDm = $derived(session.view.startsWith('dm:'));
|
||||
const title = $derived(session.status === 'connected' ? session.viewTitle() : 'mumh5');
|
||||
@@ -17,7 +28,9 @@
|
||||
});
|
||||
</script>
|
||||
|
||||
<section class="chat" aria-label="Chat">
|
||||
<section class="chat" class:video-only={videoOnly} class:log-fixed={logFixed} aria-label="Chat">
|
||||
<!-- Stacked layout: no title bar; the channel list above already names the channel -->
|
||||
{#if !videoOnly}
|
||||
<header>
|
||||
{#if showMenu}<button class="icon-btn" onclick={onmenu} aria-label="Open channels"><Icon name="menu" /></button>{/if}
|
||||
<Icon name={isDm ? 'message' : 'hash'} size={20} />
|
||||
@@ -26,8 +39,9 @@
|
||||
<span class="grow"></span>
|
||||
<button class="icon-btn" onclick={onpanel} title="Members"><Icon name="users" /></button>
|
||||
</header>
|
||||
{/if}
|
||||
|
||||
{#if showStage && session.status === 'connected'}<VoiceStage />{/if}
|
||||
{#if session.status === 'connected'}<VoiceStage always={showStage} {videoOnly} />{/if}
|
||||
|
||||
{#if session.status === 'idle' && session.disconnectInfo}
|
||||
{@const d = session.disconnectInfo}
|
||||
@@ -47,6 +61,11 @@
|
||||
{/if}
|
||||
|
||||
|
||||
<div class="logbox" style:height={logFixed ? `${logHeight}px` : undefined}>
|
||||
{#if streamAbove}
|
||||
<ResizeHandle bind:width={logHeight} edge="top" min={LOG_MIN} max={() => Math.max(LOG_MIN, window.innerHeight - 320)} initial={220}
|
||||
storageKey="mumh5.stackedChatHeight" label="Resize the chat" />
|
||||
{/if}
|
||||
<ChatLog key={session.view}>
|
||||
{#snippet empty()}
|
||||
<div class="empty">
|
||||
@@ -66,6 +85,7 @@
|
||||
</div>
|
||||
{/snippet}
|
||||
</ChatLog>
|
||||
</div>
|
||||
|
||||
{#if session.status === 'connected'}
|
||||
<Composer target={session.view} placeholder={`Message ${isDm ? '@' : '#'}${title}`} />
|
||||
@@ -73,6 +93,13 @@
|
||||
</section>
|
||||
|
||||
<style>
|
||||
/* Stacked layout: room for about five lines of chat whatever the stream above takes */
|
||||
.chat.video-only .logbox { min-height: 124px; flex-shrink: 0; }
|
||||
.logbox { position: relative; flex: 1; min-height: 0; display: flex; flex-direction: column; }
|
||||
/* Dragged to a height of its own: the stream above takes what is left */
|
||||
.chat.log-fixed .logbox { flex: none; }
|
||||
.chat.log-fixed :global(.stage.solo), .chat.log-fixed :global(.stage.solo .spot), .chat.log-fixed :global(.stage.solo .pic) { flex: 1 1 0; min-height: 0; }
|
||||
.chat.log-fixed :global(.stage.solo .spot video) { flex: 1 1 0; max-height: none; }
|
||||
.chat { position: relative; flex: 1; min-width: 0; display: flex; flex-direction: column; background: var(--bg-2); }
|
||||
header { height: 52px; flex: none; display: flex; align-items: center; gap: 8px; padding: 0 8px 0 16px; border-bottom: 1px solid var(--line); color: var(--text-dim); }
|
||||
h1 { flex: 0 1 auto; max-width: 60%; font-size: 16px; margin: 0; color: var(--text); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
<script lang="ts">
|
||||
import { ui } from '../lib/ui.svelte.ts';
|
||||
|
||||
// An image from the chat or a description, as large as the window allows. A click anywhere
|
||||
// or Escape closes it.
|
||||
const src = $derived(ui.image);
|
||||
const remote = $derived(!!src && /^https?:/i.test(src));
|
||||
</script>
|
||||
|
||||
<svelte:window onkeydown={e => { if (src && e.key === 'Escape') { e.stopPropagation(); ui.image = null; } }} />
|
||||
|
||||
{#if src}
|
||||
<div class="viewer" role="dialog" aria-modal="true" aria-label="Image">
|
||||
<button class="shade" aria-label="Close image" onclick={() => (ui.image = null)}></button>
|
||||
<img {src} alt="" />
|
||||
{#if remote}<a class="open" href={src} target="_blank" rel="noopener noreferrer">Open original</a>{/if}
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<style>
|
||||
.viewer { position: fixed; inset: 0; z-index: 300; display: flex; align-items: center; justify-content: center; padding: 16px; }
|
||||
.shade { position: absolute; inset: 0; background: rgb(0 0 0 / 0.85); cursor: zoom-out; }
|
||||
img { position: relative; max-width: 100%; max-height: 100%; object-fit: contain; pointer-events: none; box-shadow: 0 16px 48px rgb(0 0 0 / 0.6); }
|
||||
.open { position: absolute; bottom: 14px; left: 50%; transform: translateX(-50%); padding: 4px 10px; font-size: 13px; color: #fff; background: rgb(0 0 0 / 0.7); }
|
||||
</style>
|
||||
@@ -17,6 +17,8 @@
|
||||
const u = entry.authorSession != null ? session.user(entry.authorSession) : undefined;
|
||||
if (u) menu.show(e, userMenu(u));
|
||||
}
|
||||
// Full date and time, shown at the right edge while the pointer is on the message
|
||||
const full = $derived(time.toLocaleString([], { dateStyle: 'medium', timeStyle: 'medium' }));
|
||||
const hhmm = $derived(time.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' }));
|
||||
</script>
|
||||
|
||||
@@ -27,12 +29,9 @@
|
||||
</div>
|
||||
{:else}
|
||||
<article class="msg" class:grouped>
|
||||
<time class="stamp" datetime={time.toISOString()}>{full}</time>
|
||||
<div class="gutter">
|
||||
{#if grouped}
|
||||
<span class="time hover">{hhmm}</span>
|
||||
{:else}
|
||||
<span role="presentation" oncontextmenu={authorMenu}><Avatar name={entry.author} size={38} /></span>
|
||||
{/if}
|
||||
{#if !grouped}<span role="presentation" oncontextmenu={authorMenu}><Avatar name={entry.author} size={38} /></span>{/if}
|
||||
</div>
|
||||
<div class="content">
|
||||
{#if !grouped}
|
||||
@@ -64,7 +63,10 @@
|
||||
{/if}
|
||||
|
||||
<style>
|
||||
.msg { display: flex; gap: 14px; padding: 2px 16px; margin-top: 14px; }
|
||||
.msg { position: relative; display: flex; gap: 14px; padding: 2px 16px; margin-top: 14px; }
|
||||
/* Over the end of the line, with its own background so it stays readable on long messages */
|
||||
.stamp { position: absolute; top: 2px; right: 12px; padding: 1px 6px; line-height: 18px; background: var(--bg-1); border: 1px solid var(--line); visibility: hidden; pointer-events: none; }
|
||||
.msg:hover .stamp { visibility: visible; }
|
||||
.msg.grouped { margin-top: 0; }
|
||||
.msg:hover { background: rgb(255 255 255 / 0.02); }
|
||||
.gutter { width: 38px; flex: none; display: flex; justify-content: center; }
|
||||
@@ -73,8 +75,6 @@
|
||||
header strong { font-weight: 600; }
|
||||
header strong.self { color: var(--accent-hover); }
|
||||
time, .time { font-size: 11px; color: var(--text-faint); }
|
||||
.time.hover { visibility: hidden; line-height: 22px; }
|
||||
.msg:hover .time.hover { visibility: visible; }
|
||||
.html { overflow-wrap: anywhere; user-select: text; }
|
||||
.html :global(p) { margin: 0; }
|
||||
.html :global(img) { max-width: min(520px, 100%); max-height: 360px; display: block; margin-top: 4px; border: 1px solid var(--line); }
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
import { voice } from '../lib/audio/voice.svelte.ts';
|
||||
import { renderIncoming } from '../lib/html.ts';
|
||||
import RichEditor from './RichEditor.svelte';
|
||||
import { share } from '../lib/share.svelte.ts';
|
||||
import { sessions } from '../lib/session.svelte.ts';
|
||||
|
||||
let { onnavigate, onclose }: { onnavigate: () => void; onclose: () => void } = $props();
|
||||
|
||||
@@ -124,6 +126,16 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if session.sharing[user.session]}
|
||||
<div class="sharing">
|
||||
<Icon name={session.sharing[user.session]} size={16} />
|
||||
<span>{isSelf ? `You are sharing your ${session.sharing[user.session]}` : `Sharing their ${session.sharing[user.session]}`}</span>
|
||||
{#if isSelf}<button class="btn" onclick={() => share.stop()}>Stop sharing</button>
|
||||
{:else if share.watchOf(user.session)}<button class="btn" onclick={() => share.unwatch(user.session)}>Stop watching</button>
|
||||
{:else}<button class="btn primary" onclick={() => share.watch(sessions.active, user.session)}>Watch</button>{/if}
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
{#if isSelf}
|
||||
<div class="actions"><button class="btn" onclick={() => changeNamePrompt(user)}>Change name</button></div>
|
||||
{/if}
|
||||
@@ -158,6 +170,9 @@
|
||||
</aside>
|
||||
|
||||
<style>
|
||||
.sharing { display: flex; align-items: center; gap: 8px; margin-top: 12px; padding: 8px 10px; font-size: 13px; color: var(--speaking); border: 1px solid var(--line); background: var(--bg-0); }
|
||||
.sharing span { flex: 1; min-width: 0; color: var(--text); }
|
||||
.sharing .btn { padding: 4px 10px; font-size: 12px; }
|
||||
.panel { position: relative; max-width: 100%; height: 100%; flex: none; display: flex; flex-direction: column; background: var(--bg-1); border-left: 1px solid var(--line); min-height: 0; }
|
||||
header { height: 52px; flex: none; display: flex; align-items: center; gap: 8px; padding: 0 8px 0 14px; border-bottom: 1px solid var(--line); color: var(--text-dim); }
|
||||
h2 { flex: 1; min-width: 0; margin: 0; font-size: 15px; color: var(--text); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
|
||||
@@ -96,6 +96,8 @@
|
||||
</nav>
|
||||
|
||||
<style>
|
||||
/* The server you are talking on leads the list, wherever it sits in your own order */
|
||||
.tile.voice { order: -1; }
|
||||
.rail {
|
||||
width: 68px; flex: none; background: var(--bg-0); display: flex; flex-direction: column;
|
||||
align-items: center; gap: 8px; padding: 12px 0; overflow-y: auto;
|
||||
|
||||
@@ -16,6 +16,21 @@
|
||||
const s = store.settings;
|
||||
let embedHosts = $state(s.embedHosts.join('\n'));
|
||||
let testResult = $state('');
|
||||
|
||||
// Public STUN servers to pick from; anything else is typed in
|
||||
const STUN_PRESETS = [
|
||||
{ name: 'Google', url: 'stun.l.google.com:19302' },
|
||||
{ name: 'Cloudflare', url: 'stun.cloudflare.com:3478' }
|
||||
];
|
||||
let stunCustom = $state(!!s.stunServer && !STUN_PRESETS.some(p => p.url === s.stunServer));
|
||||
const stunChoice = $derived(stunCustom ? 'custom' : s.stunServer);
|
||||
function chooseStun(value: string) {
|
||||
stunCustom = value === 'custom';
|
||||
// Keep what was typed when switching to "another server"
|
||||
if (!stunCustom) s.stunServer = value;
|
||||
else if (STUN_PRESETS.some(p => p.url === s.stunServer)) s.stunServer = '';
|
||||
store.saveSettings();
|
||||
}
|
||||
// Settings that only work through the upload host are disabled until it is set up
|
||||
const hasUpload = $derived(!!(s.uploadHost.trim() && s.uploadKey.trim()));
|
||||
const needsUpload = $derived(hasUpload ? undefined : 'Needs an upload host and key');
|
||||
@@ -59,6 +74,21 @@
|
||||
|
||||
{#if tab === 'voice'}
|
||||
<VoiceSettings />
|
||||
<h3>Screen sharing</h3>
|
||||
<label for="s-stun">STUN server</label>
|
||||
<select id="s-stun" value={stunChoice} onchange={e => chooseStun(e.currentTarget.value)}>
|
||||
<option value="">{isWeb ? 'This site\'s own server' : 'None (same network only)'}</option>
|
||||
{#each STUN_PRESETS as p (p.url)}<option value={p.url}>{p.name} ({p.url})</option>{/each}
|
||||
<option value="custom">Another server...</option>
|
||||
</select>
|
||||
{#if stunChoice === 'custom'}
|
||||
<input class="stun-custom" bind:value={s.stunServer} onchange={() => store.saveSettings()} placeholder="stun.example.com:3478" autocomplete="off" aria-label="STUN server address" />
|
||||
{/if}
|
||||
<label for="s-relay">Relay <span class="opt">(optional)</span></label>
|
||||
<input id="s-relay" bind:value={s.relayServer} onchange={() => { s.relayServer = s.relayServer.trim(); store.saveSettings(); }}
|
||||
placeholder={isWeb ? 'this site (automatic)' : 'https://voice.example.com'} autocomplete="off" />
|
||||
<p class="help">The address of a mumh5 web version. When a direct connection is not possible (mobile data, strict company networks), the stream goes through that server instead, by itself. {isWeb ? 'Empty uses the site you are on.' : 'Its operator sees your IP address and carries the stream, but cannot watch it.'}</p>
|
||||
<p class="help">Screen sharing connects you directly to the people watching. Across the internet both sides need a STUN server to find a route; it learns your IP address but never sees the stream. {isWeb ? 'By default the site you are on provides it.' : 'With none, no outside server is contacted and streams only work on the same network.'} Networks that block direct connections, like many mobile ones, need more than STUN.</p>
|
||||
{:else if tab === 'sounds'}
|
||||
<SoundSettings />
|
||||
{:else if tab === 'appearance'}
|
||||
@@ -119,6 +149,8 @@
|
||||
.tabs button { padding: 8px 14px; color: var(--text-dim); font-weight: 600; white-space: nowrap; }
|
||||
.tabs button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); }
|
||||
h3 { margin: 22px 0 0; font-size: 14px; }
|
||||
.stun-custom { margin-top: 6px; }
|
||||
.opt { text-transform: none; font-weight: 400; letter-spacing: 0; color: var(--text-faint); }
|
||||
.help { margin: 10px 0 0; font-size: 13px; color: var(--text-dim); }
|
||||
.row { display: flex; align-items: flex-end; gap: 8px; }
|
||||
.row > div { flex: 1; }
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
<script lang="ts">
|
||||
import { onDestroy } from 'svelte';
|
||||
import Modal from './Modal.svelte';
|
||||
import { share, QUALITY, type ShareKind } from '../lib/share.svelte.ts';
|
||||
import { desktop, type ScreenSource } from '../lib/native.ts';
|
||||
import { store } from '../lib/settings.svelte.ts';
|
||||
|
||||
// Everything is chosen here, with a preview, before anything is sent: what to show, which
|
||||
// sound goes with it and the quality. In a browser, and on Wayland, the system's own dialog
|
||||
// picks the screen or window.
|
||||
let { onclose }: { onclose: () => void } = $props();
|
||||
const target = share.dialog!;
|
||||
|
||||
let info = $state<{ portal: boolean; audio: 'pipewire' | 'loopback' | 'browser' | 'none'; micLabel: string }>({ portal: true, audio: desktop ? 'none' : 'browser', micLabel: '' });
|
||||
// What to show: the screen (or a window), or a camera
|
||||
// Phones have no screen capture in the browser; there the camera is the only choice
|
||||
const canScreen = !!navigator.mediaDevices?.getDisplayMedia;
|
||||
let kind = $state<ShareKind>(canScreen ? 'screen' : 'camera');
|
||||
let cameras = $state<MediaDeviceInfo[]>([]);
|
||||
let camera = $state('');
|
||||
let sources = $state<ScreenSource[]>([]);
|
||||
let selected = $state('');
|
||||
let preview = $state.raw<MediaStream | null>(null);
|
||||
let apps = $state<string[]>([]);
|
||||
// '' none, 'all' everything, 'app:<name>' one program
|
||||
let audio = $state('');
|
||||
let browserAudio = $state(false);
|
||||
let busy = $state(false);
|
||||
let error = $state('');
|
||||
let handedOver = false;
|
||||
let video = $state<HTMLVideoElement>();
|
||||
$effect(() => { if (video) video.srcObject = preview; });
|
||||
|
||||
const fps = () => QUALITY[store.settings.shareQuality].fps;
|
||||
|
||||
let listing = $state(false);
|
||||
async function loadSources() {
|
||||
if (!desktop) return;
|
||||
listing = true;
|
||||
try {
|
||||
sources = (await desktop.screenSources(false)).sources;
|
||||
} finally {
|
||||
listing = false;
|
||||
}
|
||||
// Pictures follow when they are ready
|
||||
const pictures = new Map((await desktop.screenSources(true)).sources.map(s => [s.id, s.thumbnail]));
|
||||
sources = sources.map(s => ({ ...s, thumbnail: pictures.get(s.id) ?? s.thumbnail }));
|
||||
}
|
||||
const loadApps = async () => { if (desktop && info.audio === 'pipewire') apps = await desktop.shareAudioApps(); };
|
||||
|
||||
if (desktop) {
|
||||
desktop.shareInfo().then(async i => {
|
||||
info = i;
|
||||
loadApps();
|
||||
if (!i.portal) await loadSources();
|
||||
});
|
||||
}
|
||||
|
||||
function dropPreview() {
|
||||
preview?.getTracks().forEach(t => t.stop());
|
||||
preview = null;
|
||||
}
|
||||
|
||||
async function show(id: string) {
|
||||
error = '';
|
||||
dropPreview();
|
||||
selected = id;
|
||||
try {
|
||||
desktop!.pickScreenSource(id);
|
||||
preview = await navigator.mediaDevices.getDisplayMedia({ video: { frameRate: fps() }, audio: false });
|
||||
} catch (e) {
|
||||
error = `Could not capture this: ${(e as Error).message}`;
|
||||
}
|
||||
}
|
||||
|
||||
async function showCamera(deviceId = camera) {
|
||||
error = '';
|
||||
dropPreview();
|
||||
const wanted: MediaTrackConstraints = { width: { ideal: 1280 }, height: { ideal: 720 }, frameRate: { ideal: fps() } };
|
||||
if (deviceId) wanted.deviceId = { exact: deviceId };
|
||||
else wanted.facingMode = 'user';
|
||||
try {
|
||||
try {
|
||||
preview = await navigator.mediaDevices.getUserMedia({ video: wanted });
|
||||
} catch (e) {
|
||||
// A refusal or a missing camera is final; anything else may be a phone camera that
|
||||
// dislikes the wishes above, so ask once more for whatever it has
|
||||
if (['NotAllowedError', 'SecurityError', 'NotFoundError'].includes((e as Error).name)) throw e;
|
||||
preview = await navigator.mediaDevices.getUserMedia({ video: deviceId ? { deviceId: { exact: deviceId } } : true });
|
||||
}
|
||||
// Names are only given out once a camera was allowed
|
||||
cameras = (await navigator.mediaDevices.enumerateDevices()).filter(d => d.kind === 'videoinput');
|
||||
camera = preview.getVideoTracks()[0]?.getSettings().deviceId ?? deviceId;
|
||||
} catch (e) {
|
||||
const { name, message } = e as Error;
|
||||
error = name === 'NotFoundError' ? 'No camera was found.'
|
||||
: name === 'NotAllowedError' || name === 'SecurityError'
|
||||
? 'The camera was refused. If no question appeared, access is blocked: allow the camera for this site (the icon next to the address), and for the browser app in the phone\'s settings, then try again.'
|
||||
: `Could not open the camera (${name}): ${message}`;
|
||||
}
|
||||
}
|
||||
|
||||
if (!canScreen) showCamera();
|
||||
|
||||
function setKind(next: ShareKind) {
|
||||
if (kind === next) return;
|
||||
kind = next;
|
||||
selected = '';
|
||||
dropPreview();
|
||||
error = '';
|
||||
if (next === 'camera') showCamera();
|
||||
}
|
||||
|
||||
// The system's or the browser's picker
|
||||
async function chooseElsewhere() {
|
||||
error = '';
|
||||
try {
|
||||
if (desktop) {
|
||||
const { sources: picked } = await desktop.screenSources(false);
|
||||
if (!picked.length) return;
|
||||
sources = picked;
|
||||
await show(picked[0].id);
|
||||
} else {
|
||||
dropPreview();
|
||||
preview = await navigator.mediaDevices.getDisplayMedia({ video: { frameRate: fps() }, audio: browserAudio });
|
||||
}
|
||||
} catch (e) {
|
||||
if ((e as Error).name !== 'NotAllowedError') error = `Could not capture the screen: ${(e as Error).message}`;
|
||||
}
|
||||
}
|
||||
|
||||
// The virtual microphone shows up as a recording device a moment after it is created
|
||||
async function virtualMic(): Promise<MediaStreamTrack> {
|
||||
const find = async () => (await navigator.mediaDevices.enumerateDevices()).find(d => d.kind === 'audioinput' && d.label.includes(info.micLabel));
|
||||
let device = await find();
|
||||
for (let i = 0; i < 20 && !device; i++) {
|
||||
await new Promise(r => setTimeout(r, 150));
|
||||
device = await find();
|
||||
}
|
||||
if (!device) throw new Error('The audio capture device did not appear.');
|
||||
const s = await navigator.mediaDevices.getUserMedia({
|
||||
audio: { deviceId: { exact: device.deviceId }, echoCancellation: false, autoGainControl: false, noiseSuppression: false, channelCount: 2 }
|
||||
});
|
||||
return s.getAudioTracks()[0];
|
||||
}
|
||||
|
||||
async function start() {
|
||||
if (!preview) return;
|
||||
busy = true;
|
||||
error = '';
|
||||
let usesVirtualMic = false;
|
||||
try {
|
||||
// A camera comes without extra sound: your voice already goes through the voice chat
|
||||
if (kind === 'camera') { /* nothing to add */ }
|
||||
else if (audio && info.audio === 'pipewire') {
|
||||
usesVirtualMic = true;
|
||||
await desktop!.shareAudioStart(audio === 'all' ? { kind: 'all' } : { kind: 'app', name: audio.slice(4) });
|
||||
preview.addTrack(await virtualMic());
|
||||
} else if (audio && info.audio === 'loopback') {
|
||||
// System sound comes with a capture request; only its audio track is kept
|
||||
desktop!.pickScreenSource(selected);
|
||||
const extra = await navigator.mediaDevices.getDisplayMedia({ video: true, audio: true });
|
||||
extra.getVideoTracks().forEach(t => t.stop());
|
||||
const track = extra.getAudioTracks()[0];
|
||||
if (track) preview.addTrack(track);
|
||||
}
|
||||
} catch (e) {
|
||||
if (usesVirtualMic) desktop!.shareAudioStop();
|
||||
error = `Could not capture the sound: ${(e as Error).message}`;
|
||||
busy = false;
|
||||
return;
|
||||
}
|
||||
handedOver = true;
|
||||
share.go(target, preview, usesVirtualMic, kind);
|
||||
onclose();
|
||||
}
|
||||
|
||||
onDestroy(() => { if (!handedOver) dropPreview(); });
|
||||
</script>
|
||||
|
||||
<Modal title="Share your screen" {onclose} width={760}>
|
||||
<p class="help">mumh5 users in your channel can watch. Nothing is sent until you press Start sharing.</p>
|
||||
|
||||
<div class="kinds" role="tablist" aria-label="What to share">
|
||||
<button role="tab" aria-selected={kind === 'screen'} class:active={kind === 'screen'} disabled={!canScreen}
|
||||
title={canScreen ? undefined : 'This browser cannot capture the screen'} onclick={() => setKind('screen')}>Screen or window</button>
|
||||
<button role="tab" aria-selected={kind === 'camera'} class:active={kind === 'camera'} onclick={() => setKind('camera')}>Camera</button>
|
||||
</div>
|
||||
|
||||
{#if !canScreen}<p class="help">This browser cannot capture the screen (no browser on Android or iOS can), so only a camera can be shared from here. Watching other people's screens works.</p>{/if}
|
||||
{#if kind === 'camera'}
|
||||
<label for="sh-cam">Camera</label>
|
||||
<select id="sh-cam" value={camera} onchange={e => showCamera(e.currentTarget.value)} disabled={!cameras.length}>
|
||||
{#each cameras as c, i (c.deviceId)}<option value={c.deviceId}>{c.label || `Camera ${i + 1}`}</option>{:else}<option value="">{error ? 'None' : 'Asking for the camera...'}</option>{/each}
|
||||
</select>
|
||||
{:else if !desktop && info.audio === 'browser'}
|
||||
<label class="check"><input type="checkbox" bind:checked={browserAudio} disabled={!!preview} /> Include sound (the browser offers what it can share, usually a tab's sound)</label>
|
||||
{/if}
|
||||
|
||||
{#if kind === 'camera'}
|
||||
<!-- chosen above -->
|
||||
{:else if desktop && !info.portal}
|
||||
<div class="row head"><label for="sh-src">Screen or window</label><button class="link" onclick={loadSources}>Refresh</button></div>
|
||||
<ul class="sources" id="sh-src">
|
||||
{#each sources as s (s.id)}
|
||||
<li>
|
||||
<button class:active={selected === s.id} aria-pressed={selected === s.id} onclick={() => show(s.id)}>
|
||||
{#if s.thumbnail}<img src={s.thumbnail} alt="" />{:else}<span class="blank"></span>{/if}
|
||||
<span class="name">{s.name}</span>
|
||||
</button>
|
||||
</li>
|
||||
{:else}
|
||||
<li class="dim">{listing ? 'Looking for screens and windows...' : 'Nothing found. Try Refresh.'}</li>
|
||||
{/each}
|
||||
</ul>
|
||||
{:else}
|
||||
<div class="row"><button class="btn" onclick={chooseElsewhere}>{preview ? 'Choose another screen or window...' : 'Choose a screen or window...'}</button></div>
|
||||
{/if}
|
||||
|
||||
<span class="label">Preview</span>
|
||||
<div class="preview">
|
||||
<!-- svelte-ignore a11y_media_has_caption -->
|
||||
<video bind:this={video} class:mirror={kind === 'camera' && preview?.getVideoTracks()[0]?.getSettings().facingMode !== 'environment'} autoplay playsinline muted></video>
|
||||
{#if !preview}<p>Nothing chosen yet</p>{/if}
|
||||
</div>
|
||||
|
||||
<div class="row two">
|
||||
{#if desktop && kind === 'screen'}
|
||||
<div>
|
||||
<label for="sh-audio">Sound</label>
|
||||
{#if info.audio === 'pipewire'}
|
||||
<select id="sh-audio" bind:value={audio} onfocus={loadApps}>
|
||||
<option value="">No sound</option>
|
||||
<option value="all">Everything except mumh5</option>
|
||||
{#each apps as a (a)}<option value={`app:${a}`}>{a}</option>{/each}
|
||||
</select>
|
||||
{:else if info.audio === 'loopback'}
|
||||
<select id="sh-audio" bind:value={audio}>
|
||||
<option value="">No sound</option>
|
||||
<option value="all">Whole system (viewers also hear the voice chat again)</option>
|
||||
</select>
|
||||
{:else}
|
||||
<select id="sh-audio" disabled><option>Not available on this system</option></select>
|
||||
{/if}
|
||||
</div>
|
||||
{/if}
|
||||
<div>
|
||||
<label for="sh-quality">Quality</label>
|
||||
<select id="sh-quality" bind:value={store.settings.shareQuality} onchange={() => store.saveSettings()}>
|
||||
{#each Object.entries(QUALITY) as [id, q] (id)}<option value={id}>{q.label}</option>{/each}
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if error}<p class="err">{error}</p>{/if}
|
||||
{#if kind === 'camera' && error}<div class="row"><button class="btn" onclick={() => showCamera()}>Try the camera again</button></div>{/if}
|
||||
|
||||
{#snippet footer()}
|
||||
<button class="btn" onclick={onclose}>Cancel</button>
|
||||
<button class="btn primary" disabled={!preview || busy} onclick={start}>{busy ? 'Starting...' : 'Start sharing'}</button>
|
||||
{/snippet}
|
||||
</Modal>
|
||||
|
||||
<style>
|
||||
.help { margin: 8px 0 0; font-size: 13px; color: var(--text-dim); }
|
||||
.kinds { display: flex; gap: 2px; margin-top: 12px; border-bottom: 1px solid var(--line); }
|
||||
.kinds button { padding: 6px 14px; color: var(--text-dim); font-weight: 600; }
|
||||
.kinds button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); }
|
||||
.row { display: flex; align-items: center; gap: 8px; margin-top: 12px; }
|
||||
.row.head { justify-content: space-between; margin-top: 0; }
|
||||
.row.two { align-items: flex-end; gap: 12px; }
|
||||
.row.two > div { flex: 1; min-width: 0; }
|
||||
.label { display: block; font-size: 12px; font-weight: 600; text-transform: uppercase; letter-spacing: 0.04em; color: var(--text-dim); margin: 14px 0 6px; }
|
||||
.link { color: var(--link); font-size: 13px; margin-top: 8px; }
|
||||
.sources { list-style: none; margin: 0; padding: 0; display: grid; grid-template-columns: repeat(auto-fill, minmax(150px, 1fr)); gap: 8px; max-height: 190px; overflow-y: auto; }
|
||||
.sources button { width: 100%; display: flex; flex-direction: column; gap: 4px; padding: 6px; text-align: left; border: 1px solid var(--line); background: var(--bg-0); }
|
||||
.sources button:hover { border-color: var(--text-faint); }
|
||||
.sources button.active { border-color: var(--accent); box-shadow: inset 0 0 0 1px var(--accent); }
|
||||
img, .blank { width: 100%; aspect-ratio: 16 / 9; object-fit: contain; background: #000; display: block; }
|
||||
.name { font-size: 12px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.preview { position: relative; background: #000; aspect-ratio: 16 / 9; max-height: 240px; margin: 0 auto; width: 100%; }
|
||||
.preview video { width: 100%; height: 100%; object-fit: contain; display: block; }
|
||||
/* Your own camera as a mirror shows you; a camera facing away is left as it is */
|
||||
.preview video.mirror { transform: scaleX(-1); }
|
||||
.preview p { position: absolute; inset: 0; margin: 0; display: flex; align-items: center; justify-content: center; color: #aaa; font-size: 13px; }
|
||||
.check { display: flex; align-items: center; gap: 8px; text-transform: none; font-weight: 400; letter-spacing: 0; font-size: 14px; color: var(--text); margin-top: 12px; }
|
||||
.check input { width: auto; }
|
||||
.dim { color: var(--text-dim); font-size: 13px; }
|
||||
.err { color: var(--danger); font-size: 13px; margin: 10px 0 0; }
|
||||
</style>
|
||||
@@ -0,0 +1,119 @@
|
||||
<script lang="ts">
|
||||
import { onDestroy } from 'svelte';
|
||||
import { share } from '../lib/share.svelte.ts';
|
||||
import { session } from '../lib/session.svelte.ts';
|
||||
import { store } from '../lib/settings.svelte.ts';
|
||||
|
||||
// "Stats for nerds" over the stream: codecs, picture, bitrate and the network path, read
|
||||
// from the WebRTC connection once a second. own: our stream (one block per viewer).
|
||||
// session: whose stream, when it is one we watch
|
||||
let { own, session: watched = null, onclose }: { own: boolean; session?: number | null; onclose: () => void } = $props();
|
||||
|
||||
type Row = [string, string];
|
||||
let blocks = $state<{ title: string; rows: Row[] }[]>([]);
|
||||
// Byte counters from the previous reading, for the bitrate
|
||||
const last = new Map<string, { bytes: number; at: number }>();
|
||||
|
||||
const kbit = (key: string, bytes: number | undefined, at: number): string => {
|
||||
if (bytes == null) return 'n/a';
|
||||
const prev = last.get(key);
|
||||
last.set(key, { bytes, at });
|
||||
if (!prev || at <= prev.at) return '...';
|
||||
return `${Math.round(((bytes - prev.bytes) * 8) / (at - prev.at))} kbit/s`;
|
||||
};
|
||||
const ms = (seconds: number | undefined) => seconds == null ? 'n/a' : `${Math.round(seconds * 1000)} ms`;
|
||||
const codecName = (report: RTCStatsReport, id?: string): string => {
|
||||
const c = id ? report.get(id) : null;
|
||||
if (!c) return 'n/a';
|
||||
const profile = /profile-level-id=([0-9a-f]+)/i.exec(c.sdpFmtpLine ?? '')?.[1];
|
||||
const stereo = /stereo=1/.test(c.sdpFmtpLine ?? '') ? ', stereo' : '';
|
||||
return `${String(c.mimeType).split('/')[1]}${profile ? ` (${profile})` : ''}${c.clockRate && c.mimeType.startsWith('audio') ? `, ${c.clockRate / 1000} kHz${stereo}` : ''}`;
|
||||
};
|
||||
const KIND: Record<string, string> = { host: 'direct', srflx: 'direct, through a router (STUN)', prflx: 'direct, through a router', relay: 'relayed (TURN)' };
|
||||
|
||||
function describe(report: RTCStatsReport, key: string): Row[] {
|
||||
const all = [...report.values()];
|
||||
const rows: Row[] = [];
|
||||
const rtp = (type: string, kind: string) => all.find(s => s.type === type && s.kind === kind);
|
||||
const video = rtp(own ? 'outbound-rtp' : 'inbound-rtp', 'video');
|
||||
const audio = rtp(own ? 'outbound-rtp' : 'inbound-rtp', 'audio');
|
||||
if (video) {
|
||||
rows.push(['Video codec', codecName(report, video.codecId)]);
|
||||
if (video.frameWidth) rows.push(['Picture', `${video.frameWidth}x${video.frameHeight}${video.framesPerSecond != null ? `, ${Math.round(video.framesPerSecond)} fps` : ''}`]);
|
||||
rows.push(['Video bitrate', kbit(`${key}v`, own ? video.bytesSent : video.bytesReceived, video.timestamp)]);
|
||||
if (own) {
|
||||
if (video.encoderImplementation) rows.push(['Encoder', video.encoderImplementation]);
|
||||
if (video.qualityLimitationReason && video.qualityLimitationReason !== 'none') rows.push(['Held back by', video.qualityLimitationReason]);
|
||||
} else {
|
||||
if (video.decoderImplementation) rows.push(['Decoder', video.decoderImplementation]);
|
||||
rows.push(['Frames dropped', `${video.framesDropped ?? 0} of ${video.framesReceived ?? 0}`]);
|
||||
rows.push(['Video packets lost', `${video.packetsLost ?? 0} of ${(video.packetsReceived ?? 0) + (video.packetsLost ?? 0)}`]);
|
||||
rows.push(['Video jitter', ms(video.jitter)]);
|
||||
}
|
||||
}
|
||||
if (audio) {
|
||||
rows.push(['Audio codec', codecName(report, audio.codecId)]);
|
||||
rows.push(['Audio bitrate', kbit(`${key}a`, own ? audio.bytesSent : audio.bytesReceived, audio.timestamp)]);
|
||||
if (!own) rows.push(['Audio packets lost', String(audio.packetsLost ?? 0)]);
|
||||
} else rows.push(['Audio', 'none']);
|
||||
if (own) {
|
||||
// What the viewer reports back about our packets
|
||||
const back = all.find(s => s.type === 'remote-inbound-rtp' && s.kind === 'video');
|
||||
if (back) rows.push(['Lost on the way', `${Math.round((back.fractionLost ?? 0) * 100)}%`]);
|
||||
}
|
||||
const transport = all.find(s => s.type === 'transport');
|
||||
const pair = (transport?.selectedCandidatePairId && report.get(transport.selectedCandidatePairId)) || all.find(s => s.type === 'candidate-pair' && s.nominated && s.state === 'succeeded');
|
||||
if (pair) {
|
||||
const local = report.get(pair.localCandidateId), remote = report.get(pair.remoteCandidateId);
|
||||
// What this side had to offer. A route discovered during the checks (prflx) is still a
|
||||
// relayed one when relays were all we had.
|
||||
const mine = all.filter(s => s.type === 'local-candidate' && s.candidateType !== 'prflx');
|
||||
const relays = mine.filter(s => s.candidateType === 'relay');
|
||||
const relayed = local?.candidateType === 'relay' || (local?.candidateType === 'prflx' && relays.length > 0 && relays.length === mine.length);
|
||||
const relayVia = (local?.relayProtocol ?? relays[0]?.relayProtocol) as string | undefined;
|
||||
// A relayed route is UDP beyond the relay; what matters is how we reach the relay
|
||||
rows.push(['Connection', relayed ? `${KIND.relay}${relayVia ? `, reached over ${relayVia.toUpperCase()}` : ''}`
|
||||
: `${KIND[local?.candidateType] ?? local?.candidateType ?? 'n/a'}, ${String(local?.protocol ?? '').toUpperCase()}`]);
|
||||
const count = (type: string) => mine.filter(s => s.candidateType === type).length;
|
||||
rows.push(['Routes offered', `${count('host')} direct, ${count('srflx')} through STUN, ${relays.length} relayed`]);
|
||||
if (remote?.candidateType) rows.push(['Other side', KIND[remote.candidateType] ?? remote.candidateType]);
|
||||
rows.push(['Round trip', ms(pair.currentRoundTripTime)]);
|
||||
if (pair.availableOutgoingBitrate) rows.push(['Estimated capacity', `${Math.round(pair.availableOutgoingBitrate / 1000)} kbit/s`]);
|
||||
} else rows.push(['Connection', 'not established']);
|
||||
return rows;
|
||||
}
|
||||
|
||||
async function read() {
|
||||
const reports = await share.reports(own, watched ?? undefined);
|
||||
blocks = reports.map(r => {
|
||||
const user = r.viewer != null ? session.user(r.viewer) : undefined;
|
||||
return { title: r.viewer == null ? '' : `To ${user ? store.displayName(user) : `#${r.viewer}`}`, rows: describe(r.report, String(r.viewer)) };
|
||||
});
|
||||
}
|
||||
read();
|
||||
const timer = setInterval(read, 1000);
|
||||
onDestroy(() => clearInterval(timer));
|
||||
</script>
|
||||
|
||||
<div class="nerd" role="status" aria-label="Stream statistics">
|
||||
<button class="close" onclick={onclose} aria-label="Close statistics">x</button>
|
||||
{#each blocks as b (b.title)}
|
||||
{#if b.title}<strong>{b.title}</strong>{/if}
|
||||
<dl>{#each b.rows as [k, v] (k)}<dt>{k}</dt><dd>{v}</dd>{/each}</dl>
|
||||
{:else}
|
||||
<p>{own ? 'Nobody is watching yet.' : 'Not connected.'}</p>
|
||||
{/each}
|
||||
</div>
|
||||
|
||||
<style>
|
||||
.nerd { position: absolute; top: 8px; left: 8px; z-index: 2; max-width: calc(100% - 16px); max-height: calc(100% - 16px); overflow: auto; padding: 8px 28px 8px 10px;
|
||||
background: rgb(0 0 0 / 0.78); color: #e8e8e8; font: 11px/1.5 var(--mono); user-select: text; }
|
||||
dl { display: grid; grid-template-columns: auto 1fr; gap: 0 12px; margin: 0; }
|
||||
dt { color: #9aa; white-space: nowrap; }
|
||||
dd { margin: 0; overflow-wrap: anywhere; }
|
||||
strong { display: block; margin: 6px 0 2px; color: #fff; }
|
||||
strong:first-of-type { margin-top: 0; }
|
||||
p { margin: 0; }
|
||||
.close { position: absolute; top: 2px; right: 4px; padding: 2px 6px; color: #ccc; font: inherit; }
|
||||
.close:hover { color: #fff; }
|
||||
</style>
|
||||
@@ -9,6 +9,7 @@
|
||||
import { PERM } from '../core/client.ts';
|
||||
import ResizeHandle from './ResizeHandle.svelte';
|
||||
import { voice } from '../lib/audio/voice.svelte.ts';
|
||||
import { share } from '../lib/share.svelte.ts';
|
||||
|
||||
// resizable: false in the phone drawer, which keeps a fixed width
|
||||
// handleEdge: which side the resize handle sits on (left when the list is on the right, classic layout)
|
||||
@@ -50,6 +51,14 @@
|
||||
ui.panelOpen = true;
|
||||
}
|
||||
|
||||
// Sharing runs on one server at a time; the button stops it from anywhere
|
||||
const sharingHere = $derived(!!share.stream);
|
||||
const canShare = $derived(session.status === 'connected' && share.supported(sessions.active));
|
||||
function toggleShare() {
|
||||
if (share.stream) share.stop();
|
||||
else share.start(sessions.active);
|
||||
}
|
||||
|
||||
// One button for the whole tree: open everything, then close everything
|
||||
let allOpen = $state(false);
|
||||
function toggleAll() {
|
||||
@@ -159,6 +168,10 @@
|
||||
onclick={() => session.setSelfDeaf(!deafened)} oncontextmenu={e => openQuick(e, 'output')}>
|
||||
<Icon name={deafened ? 'headphones-off' : 'headphones'} size={18} />
|
||||
</button>
|
||||
<button class="icon-btn" class:live={sharingHere} title={sharingHere ? `Stop sharing your ${share.kind}` : 'Share your screen or camera with your channel'}
|
||||
aria-label={sharingHere ? `Stop sharing your ${share.kind}` : 'Share your screen or camera'} disabled={!sharingHere && !canShare} onclick={toggleShare}>
|
||||
<Icon name={sharingHere ? share.kind : 'screen'} size={18} />
|
||||
</button>
|
||||
<span class="sep"></span>
|
||||
<button class="icon-btn" title="Change your description" aria-label="Change your description" disabled={!self} onclick={editDescription}>
|
||||
<Icon name="message" size={18} />
|
||||
@@ -216,5 +229,6 @@
|
||||
.who strong, .who span { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.who span { font-size: 12px; color: var(--text-dim); }
|
||||
.info, .info:hover { color: var(--warn); }
|
||||
.live, .live:hover { color: var(--speaking); }
|
||||
.offline-who { flex: 1 1 110px; padding: 2px 2px 2px 6px; color: var(--text-dim); }
|
||||
</style>
|
||||
|
||||
+410
-31
@@ -7,11 +7,117 @@
|
||||
import { ui } from '../lib/ui.svelte.ts';
|
||||
import { userMenu } from '../lib/actions.ts';
|
||||
import { voice } from '../lib/audio/voice.svelte.ts';
|
||||
import { share } from '../lib/share.svelte.ts';
|
||||
import { sessions } from '../lib/session.svelte.ts';
|
||||
import ResizeHandle from './ResizeHandle.svelte';
|
||||
import ShareStats from './ShareStats.svelte';
|
||||
|
||||
// People in your voice channel as tiles. Shown on windows too narrow for the member list
|
||||
// (always), and on any window while someone in the channel shares something. A screen is shown
|
||||
// large above the tiles, which become a strip; cameras play in the tiles themselves, as many
|
||||
// as are on, and a click enlarges one.
|
||||
// videoOnly: just the picture, as wide as the chat (stacked layout, where the channel list
|
||||
// above already shows who is here)
|
||||
let { always = false, videoOnly = false }: { always?: boolean; videoOnly?: boolean } = $props();
|
||||
|
||||
// People in your voice channel as tiles, for windows too narrow for the member list
|
||||
const self = $derived(session.self);
|
||||
const channel = $derived(self ? session.channel(self.channelId) : undefined);
|
||||
const people = $derived(self ? session.usersIn(self.channelId) : []);
|
||||
const inChannel = $derived(self ? session.usersIn(self.channelId) : []);
|
||||
const here = (s: { server?: { id: string } | null } | null | undefined) => !!s && s.server?.id === session.server?.id;
|
||||
|
||||
const watches = $derived(share.watches.filter(w => here(w.host)));
|
||||
// The screen being watched; cameras are watched next to it
|
||||
const watching = $derived(watches.find(w => w.kind === 'screen') ?? null);
|
||||
const watchOf = (s: number) => watches.find(w => w.session === s);
|
||||
const mine = $derived(share.stream && here(share.host) ? share.stream : null);
|
||||
const anySharing = $derived(inChannel.some(u => session.sharing[u.session]));
|
||||
const anyCamera = $derived(inChannel.some(u => session.sharing[u.session] === 'camera' && u.session !== self?.session));
|
||||
const visible = $derived(!!self && (videoOnly ? watches.length > 0 || !!mine : always || anySharing || watches.length > 0 || !!mine));
|
||||
|
||||
// The picture we have of someone: our own stream, or one we are watching
|
||||
function streamOf(s: number): MediaStream | null {
|
||||
if (s === self?.session) return mine;
|
||||
return watchOf(s)?.stream ?? null;
|
||||
}
|
||||
// A camera plays in the person's tile; a screen never does
|
||||
const camOf = (s: number) => session.sharing[s] === 'camera' ? streamOf(s) : null;
|
||||
|
||||
// People with a stream we can show come first, then others who share
|
||||
const rank = (s: number) => streamOf(s) ? 0 : session.sharing[s] ? 1 : 2;
|
||||
const people = $derived([...inChannel].sort((a, b) => rank(a.session) - rank(b.session)));
|
||||
// Who gets a tile: everyone, or in the stacked layout only the people whose camera we see
|
||||
const shown = $derived(videoOnly ? people.filter(u => camOf(u.session)) : people);
|
||||
|
||||
// Tile on the large stage
|
||||
let focus = $state<number | null>(null);
|
||||
const focused = $derived(focus != null ? streamOf(focus) : null);
|
||||
const focusUser = $derived(focus != null ? session.user(focus) : undefined);
|
||||
// Screen or camera, for the wording and the icons
|
||||
const focusKind = $derived((focus != null && session.sharing[focus]) || 'screen');
|
||||
// A stream we just started watching goes to the stage; it leaves when the stream ends
|
||||
let lastWatch: number | null = null;
|
||||
$effect(() => {
|
||||
const now = watching?.session ?? null;
|
||||
if (now !== lastWatch) {
|
||||
if (now != null) focus = now;
|
||||
else if (focus === lastWatch) focus = null;
|
||||
lastWatch = now;
|
||||
}
|
||||
});
|
||||
// Our own stream opens large when it starts, unless another one is on the stage
|
||||
let hadMine = false;
|
||||
$effect(() => {
|
||||
const has = !!mine;
|
||||
if (has && !hadMine && focus == null && self && share.kind === 'screen') focus = self.session;
|
||||
else if (!has && focus === self?.session) focus = null;
|
||||
hadMine = has;
|
||||
});
|
||||
// Without tiles there is nothing to choose: the watched stream, else our own
|
||||
let lastSolo: number | null = null;
|
||||
$effect(() => {
|
||||
if (!videoOnly) return;
|
||||
const solo = watching ? watching.session : mine && share.kind === 'screen' ? self?.session ?? null : null;
|
||||
if (solo !== lastSolo) { focus = solo; lastSolo = solo; }
|
||||
});
|
||||
|
||||
// The stage is as tall as the user drags it; the tiles grow and shrink to fill it. The
|
||||
// meeting view (one picture large) remembers its own height.
|
||||
const stored = (key: string, fallback: number) => { try { return Number(localStorage.getItem(key)) || fallback; } catch { return fallback; } };
|
||||
let tilesHeight = $state(stored('mumh5.stageHeight', 210));
|
||||
let meetingHeight = $state(stored('mumh5.stageHeightMeeting', 460));
|
||||
const maxHeight = () => Math.max(160, window.innerHeight - 220);
|
||||
|
||||
// The tiles fill the stage completely, like a meeting's tile view: rows of equal tiles, with
|
||||
// the number of columns chosen so the tiles come closest to a 16:9 shape. A last row with
|
||||
// fewer people stretches, so no space is left over. Measured live, so it follows every resize.
|
||||
let gridW = $state(0), gridH = $state(0);
|
||||
const GAP = 8, PAD = 8;
|
||||
const cols = $derived.by(() => {
|
||||
const n = shown.length;
|
||||
const w = gridW - PAD * 2, h = gridH - PAD * 2;
|
||||
if (n < 2 || w <= 0 || h <= 0) return 1;
|
||||
let best = 1, bestSize = 0;
|
||||
for (let c = 1; c <= n; c++) {
|
||||
const r = Math.ceil(n / c);
|
||||
// Width of the largest 16:9 picture that fits one cell
|
||||
const size = Math.min((w - GAP * (c - 1)) / c, ((h - GAP * (r - 1)) / r) * 16 / 9);
|
||||
if (size > bestSize + 0.5) { best = c; bestSize = size; }
|
||||
}
|
||||
return best;
|
||||
});
|
||||
// In the meeting view everyone sits in one row under the picture
|
||||
const rows = $derived.by(() => {
|
||||
if (focused) return [shown];
|
||||
const out: (typeof shown)[] = [];
|
||||
for (let i = 0; i < shown.length; i += cols) out.push(shown.slice(i, i + cols));
|
||||
return out;
|
||||
});
|
||||
const avatar = $derived.by(() => {
|
||||
if (focused) return 36;
|
||||
const cellW = (gridW - PAD * 2 - GAP * (cols - 1)) / cols;
|
||||
const cellH = (gridH - PAD * 2 - GAP * (rows.length - 1)) / Math.max(1, rows.length);
|
||||
return Math.max(24, Math.min(128, Math.round(Math.min(cellH * 0.5, cellW * 0.34))));
|
||||
});
|
||||
|
||||
let collapsed = $state(load());
|
||||
function load(): boolean {
|
||||
@@ -23,10 +129,235 @@
|
||||
}
|
||||
|
||||
const talking = (s: number) => !!voice.talking[s] || (s === self?.session && voice.transmitting);
|
||||
|
||||
// Mobile browsers do not always honour autoplay for a stream attached later, so play is asked for too
|
||||
function media(node: HTMLMediaElement, stream: MediaStream | null) {
|
||||
const attach = (s: MediaStream | null) => {
|
||||
node.srcObject = s;
|
||||
if (s) node.play().catch(() => {});
|
||||
};
|
||||
attach(stream);
|
||||
return { update(next: MediaStream | null) { if (node.srcObject !== next) attach(next); } };
|
||||
}
|
||||
|
||||
// Sound comes from hidden players, one per stream that has any, so it plays once however
|
||||
// many pictures are shown. They all follow the stream volume.
|
||||
const withSound = $derived(watches.filter(w => w.stream?.getAudioTracks().length));
|
||||
const hasSound = $derived(focus != null && withSound.some(w => w.session === focus));
|
||||
function loud(node: HTMLMediaElement) {
|
||||
$effect(() => {
|
||||
node.volume = store.settings.shareVolume;
|
||||
node.muted = store.settings.shareVolume === 0;
|
||||
});
|
||||
}
|
||||
const focusWatch = $derived(focus != null ? watchOf(focus) : undefined);
|
||||
|
||||
function tileClick(session_: number) {
|
||||
if (streamOf(session_)) focus = focus === session_ ? null : session_;
|
||||
else if (session.sharing[session_] && session_ !== self?.session) share.watch(sessions.active, session_);
|
||||
else { session.showUser(session_); ui.panelOpen = true; }
|
||||
}
|
||||
|
||||
// Right-click on the picture
|
||||
let showStats = $state(false);
|
||||
const streamItems = () => [
|
||||
{ label: 'Open in a separate window', action: popOut },
|
||||
{ label: 'Full screen', action: fullscreen },
|
||||
{ separator: true, label: '' },
|
||||
{ label: 'Stats for nerds', checked: showStats, action: () => (showStats = !showStats) }
|
||||
];
|
||||
const streamMenu = (e: MouseEvent) => menu.show(e, streamItems());
|
||||
// Touch screens have no right-click: holding a finger on the picture opens the same menu
|
||||
let hold: ReturnType<typeof setTimeout> | undefined;
|
||||
function touchStart(e: TouchEvent) {
|
||||
const touch = e.touches[0];
|
||||
clearTimeout(hold);
|
||||
if (e.touches.length !== 1) return;
|
||||
const { clientX, clientY } = touch;
|
||||
hold = setTimeout(() => menu.showAt(clientX, clientY, streamItems()), 550);
|
||||
}
|
||||
const touchEnd = () => clearTimeout(hold);
|
||||
|
||||
let frame = $state<HTMLElement>();
|
||||
function fullscreen() {
|
||||
if (document.fullscreenElement) document.exitFullscreen();
|
||||
else frame?.requestFullscreen().catch(() => {});
|
||||
}
|
||||
|
||||
// The large picture in a window of its own, to put on another screen. The window is an
|
||||
// empty page that gets the same stream; sound keeps coming from the main window.
|
||||
let popup: Window | null = null;
|
||||
let popVideo: HTMLVideoElement | null = null;
|
||||
let spotVideo = $state<HTMLVideoElement>();
|
||||
let popped = $state(false);
|
||||
function popOut() {
|
||||
if (popup && !popup.closed) return popup.focus();
|
||||
const w = window.open('', 'mumh5-stream', 'popup,width=960,height=600');
|
||||
if (!w) {
|
||||
// Refused (a pop-up blocker, or an app build that predates this). The floating
|
||||
// picture-in-picture window needs no permission where the browser has it.
|
||||
if (document.pictureInPictureEnabled && spotVideo) {
|
||||
spotVideo.requestPictureInPicture().catch(() => { share.error = 'The stream could not be opened in a separate window.'; });
|
||||
} else {
|
||||
share.error = 'The stream could not be opened in a separate window. If your browser blocked a pop-up, allow it for this page and try again.';
|
||||
}
|
||||
return;
|
||||
}
|
||||
popup = w;
|
||||
popped = true;
|
||||
w.document.body.style.cssText = 'margin:0;background:#000;overflow:hidden';
|
||||
const v = w.document.createElement('video');
|
||||
v.autoplay = true;
|
||||
v.muted = true;
|
||||
v.playsInline = true;
|
||||
v.style.cssText = 'width:100vw;height:100vh;object-fit:contain;display:block';
|
||||
v.ondblclick = () => { if (w.document.fullscreenElement) w.document.exitFullscreen(); else v.requestFullscreen().catch(() => {}); };
|
||||
w.document.body.replaceChildren(v);
|
||||
popVideo = v;
|
||||
w.addEventListener('pagehide', () => { if (popup === w) { popup = null; popVideo = null; popped = false; } });
|
||||
}
|
||||
function popIn() {
|
||||
popup?.close();
|
||||
popup = null;
|
||||
popVideo = null;
|
||||
popped = false;
|
||||
}
|
||||
// The window follows the picture on the stage and closes with it
|
||||
$effect(() => {
|
||||
const stream = focused;
|
||||
const title = focus === self?.session ? `Your ${focusKind}` : focusUser ? store.displayName(focusUser) : 'Screen share';
|
||||
if (!popped) return;
|
||||
if (!stream) return popIn();
|
||||
if (popVideo && popVideo.srcObject !== stream) popVideo.srcObject = stream;
|
||||
if (popup) popup.document.title = `${title} - mumh5`;
|
||||
});
|
||||
$effect(() => () => popup?.close());
|
||||
</script>
|
||||
|
||||
{#if self}
|
||||
<section class="stage" aria-label="Voice channel">
|
||||
{#if voice.error && session.status === 'connected'}<div class="bar bad" role="alert"><span>Voice is not working: {voice.error}</span></div>{/if}
|
||||
{#if share.error}<div class="bar bad" role="alert"><span>{share.error}</span><button class="btn" onclick={() => (share.error = '')}>Dismiss</button></div>{/if}
|
||||
|
||||
{#snippet switcher()}
|
||||
{#if mine && share.kind === 'camera' && share.cameras > 1}
|
||||
<button class="icon-btn" title="Switch to the next camera" aria-label="Switch camera" disabled={share.switching} onclick={() => share.switchCamera()}>
|
||||
<Icon name="switch" size={18} />
|
||||
</button>
|
||||
{/if}
|
||||
{/snippet}
|
||||
|
||||
{#snippet spot()}
|
||||
{#if self && focus != null && focused}
|
||||
<div class="spot" role="region" aria-label="Screen share" bind:this={frame}>
|
||||
<div class="spot-head">
|
||||
<Icon name={focusKind} size={16} />
|
||||
<span class="spot-who">{focus === self.session ? `Your ${focusKind}` : focusUser ? store.displayName(focusUser) : 'Screen share'}</span>
|
||||
{#if focus !== self.session && hasSound}
|
||||
<Icon name={store.settings.shareVolume === 0 ? 'headphones-off' : 'headphones'} size={16} />
|
||||
<input class="vol" type="range" min="0" max="1" step="0.01" bind:value={store.settings.shareVolume} onchange={() => store.saveSettings()}
|
||||
aria-label="Stream volume" title="Stream volume" />
|
||||
{/if}
|
||||
{#if popped}
|
||||
<button class="btn small" onclick={popIn}>Bring back</button>
|
||||
{:else}
|
||||
<button class="icon-btn" title="Open in a separate window" aria-label="Open in a separate window" onclick={popOut}><Icon name="popout" size={16} /></button>
|
||||
<button class="icon-btn" title="Full screen" aria-label="Full screen" onclick={fullscreen}><Icon name="expand" size={16} /></button>
|
||||
{/if}
|
||||
{#if focus !== self.session}<button class="btn small" onclick={() => { if (focus != null) share.unwatch(focus); }}>Stop watching</button>
|
||||
{:else if videoOnly}
|
||||
{@render switcher()}
|
||||
<span class="status" role="status" title="People watching your {share.kind}">{share.viewers} watching</span>
|
||||
<button class="btn small" onclick={() => share.stop()}>Stop sharing</button>
|
||||
{/if}
|
||||
{#if !videoOnly || shown.length}<button class="icon-btn" title="Back to tiles" aria-label="Back to tiles" onclick={() => (focus = null)}><Icon name="x" size={16} /></button>{/if}
|
||||
</div>
|
||||
<!-- svelte-ignore a11y_media_has_caption -->
|
||||
{#if popped}
|
||||
<p class="away">Shown in a separate window.</p>
|
||||
{:else}
|
||||
<div class="pic" role="presentation" oncontextmenu={streamMenu}
|
||||
ontouchstart={touchStart} ontouchend={touchEnd} ontouchmove={touchEnd} ontouchcancel={touchEnd}>
|
||||
<!-- Large, your own camera is shown as the others see it; only the small tile is a mirror -->
|
||||
<video bind:this={spotVideo} use:media={focused} autoplay playsinline muted ondblclick={fullscreen}></video>
|
||||
{#if focus !== self.session && focusWatch && focusWatch.state !== 'live'}
|
||||
<p class="pending" class:bad={focusWatch.state === 'failed'}>{focusWatch.state === 'failed'
|
||||
? 'Could not connect to the stream. Across the internet both of you need a STUN server; networks that block direct connections need a relay. Both are in Settings, Voice.'
|
||||
: 'Connecting to the stream...'}</p>
|
||||
{/if}
|
||||
{#if showStats}<ShareStats own={focus === self.session} session={focus} onclose={() => (showStats = false)} />{/if}
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
{/if}
|
||||
{/snippet}
|
||||
|
||||
{#snippet tiles()}
|
||||
<div class="tiles" class:strip={!!focused} role="group" aria-label="People in this channel" bind:clientWidth={gridW} bind:clientHeight={gridH}>
|
||||
{#each rows as row, r (r)}
|
||||
<div class="row" role="presentation">
|
||||
{#each row as u (u.session)}
|
||||
{@const stream = streamOf(u.session)}
|
||||
{@const sharing = session.sharing[u.session]}
|
||||
{@const isSelf = u.session === self?.session}
|
||||
<!-- While a camera is shown large, its tile goes back to the avatar: one picture, not two -->
|
||||
{@const cam = focus === u.session ? null : camOf(u.session)}
|
||||
{@const w = watchOf(u.session)}
|
||||
<button class="tile" class:talking={talking(u.session)} class:self={isSelf} class:focused={focus === u.session}
|
||||
onclick={() => tileClick(u.session)}
|
||||
oncontextmenu={e => menu.show(e, userMenu(u))}
|
||||
title={stream ? (focus === u.session ? 'Hide the stream' : 'Show the stream') : sharing && !isSelf ? `Sharing their ${sharing}. Click to watch` : store.displayName(u)}
|
||||
aria-label={stream ? `${focus === u.session ? 'Hide' : 'Show'} the ${sharing || 'stream'} of ${store.displayName(u)}` : sharing && !isSelf ? `Watch the ${sharing} of ${store.displayName(u)}` : store.displayName(u)}>
|
||||
{#if cam}
|
||||
<!-- svelte-ignore a11y_media_has_caption -->
|
||||
<video class:mirror={isSelf && share.mirror} use:media={cam} autoplay playsinline muted></video>
|
||||
{:else}
|
||||
<Avatar name={u.name} size={avatar} speaking={talking(u.session)} />
|
||||
{/if}
|
||||
<span class="who">{store.displayName(u)}</span>
|
||||
{#if w && w.state !== 'live'}
|
||||
<span class="note">{w.state === 'connecting' ? 'Connecting...' : 'Could not connect'}</span>
|
||||
{/if}
|
||||
{#if sharing}<span class="badge live"><Icon name={sharing} size={13} /></span>
|
||||
{:else if u.deaf || u.selfDeaf}
|
||||
<span class="badge" class:admin={u.deaf}><Icon name="headphones-off" size={13} /></span>
|
||||
{:else if u.mute || u.selfMute || u.suppress}
|
||||
<span class="badge" class:admin={u.mute || u.suppress}><Icon name="mic-off" size={13} /></span>
|
||||
{/if}
|
||||
</button>
|
||||
{/each}
|
||||
</div>
|
||||
{/each}
|
||||
</div>
|
||||
{/snippet}
|
||||
|
||||
{#if visible && self && videoOnly}
|
||||
<section class="stage solo" aria-label="Voice channel">
|
||||
{#each withSound as w (w.session)}<audio use:media={w.stream} use:loud autoplay></audio>{/each}
|
||||
{#if focused}
|
||||
{@render spot()}
|
||||
{:else if watching}
|
||||
<p class="wait" class:bad={watching.state === 'failed'}>
|
||||
<span>{watching.state === 'failed' ? 'Could not connect to the stream. Across the internet both of you need a STUN server set in Settings, Voice.' : 'Connecting to the stream...'}</span>
|
||||
<button class="btn small" onclick={() => share.unwatch(watching.session)}>{watching.state === 'failed' ? 'Close' : 'Cancel'}</button>
|
||||
</p>
|
||||
{/if}
|
||||
<!-- Cameras: a grid of just the pictures -->
|
||||
{#if shown.length && !focused}
|
||||
<div class="camgrid" style:height="{tilesHeight}px">
|
||||
<ResizeHandle bind:width={tilesHeight} edge="bottom" min={110} max={maxHeight} initial={210} storageKey="mumh5.stageHeight" label="Resize the tiles" />
|
||||
{@render tiles()}
|
||||
</div>
|
||||
{/if}
|
||||
</section>
|
||||
{:else if visible && self}
|
||||
<section class="stage" class:meeting={!!focused} class:collapsed aria-label="Voice channel"
|
||||
style:height={collapsed ? undefined : `${focused ? meetingHeight : tilesHeight}px`}>
|
||||
{#if !collapsed}
|
||||
{#if focused}
|
||||
<ResizeHandle bind:width={meetingHeight} edge="bottom" min={240} max={maxHeight} initial={460} storageKey="mumh5.stageHeightMeeting" label="Resize the meeting view" />
|
||||
{:else}
|
||||
<ResizeHandle bind:width={tilesHeight} edge="bottom" min={110} max={maxHeight} initial={210} storageKey="mumh5.stageHeight" label="Resize the tiles" />
|
||||
{/if}
|
||||
{/if}
|
||||
<header>
|
||||
<button class="title" onclick={toggle} aria-expanded={!collapsed}>
|
||||
<svg class="caret" class:collapsed width="10" height="10" viewBox="0 0 10 10" aria-hidden="true"><path d="M2 3l3 4 3-4" fill="none" stroke="currentColor" stroke-width="1.6" /></svg>
|
||||
@@ -38,6 +369,17 @@
|
||||
</span>
|
||||
{/if}
|
||||
</button>
|
||||
{#if anyCamera || share.cams}
|
||||
<button class="icon-btn" class:live={share.cams} aria-pressed={share.cams} title={share.cams ? 'Hide everyone\'s camera' : 'Show everyone\'s camera'}
|
||||
aria-label={share.cams ? 'Hide all cameras' : 'Show all cameras'} onclick={() => share.setCams(sessions.active, !share.cams)}>
|
||||
<Icon name="camera" size={18} />
|
||||
</button>
|
||||
{/if}
|
||||
{#if mine}
|
||||
{@render switcher()}
|
||||
<span class="status" role="status" title="People watching your {share.kind}"><Icon name={share.kind} size={14} />{share.viewers} watching</span>
|
||||
<button class="btn small" onclick={() => share.stop()}>Stop sharing</button>
|
||||
{/if}
|
||||
<button class="icon-btn" class:on={self.selfMute} aria-label={self.selfMute ? 'Unmute' : 'Mute'} title={self.selfMute ? 'Unmute' : 'Mute'}
|
||||
onclick={() => session.setSelfMute(!self.selfMute)}>
|
||||
<Icon name={self.selfMute ? 'mic-off' : 'mic'} size={18} />
|
||||
@@ -47,31 +389,23 @@
|
||||
<Icon name={self.selfDeaf ? 'headphones-off' : 'headphones'} size={18} />
|
||||
</button>
|
||||
</header>
|
||||
|
||||
{#each withSound as w (w.session)}<audio use:media={w.stream} use:loud autoplay></audio>{/each}
|
||||
|
||||
{#if focused && focus != null && !collapsed}{@render spot()}{/if}
|
||||
|
||||
{#if !collapsed}
|
||||
<ul class="tiles">
|
||||
{#each people as u (u.session)}
|
||||
<li>
|
||||
<button class="tile" class:talking={talking(u.session)} class:self={u.session === self.session}
|
||||
onclick={() => { session.showUser(u.session); ui.panelOpen = true; }}
|
||||
oncontextmenu={e => menu.show(e, userMenu(u))} title={store.displayName(u)}>
|
||||
<Avatar name={u.name} size={44} speaking={talking(u.session)} />
|
||||
<span class="who">{store.displayName(u)}</span>
|
||||
{#if u.deaf || u.selfDeaf}
|
||||
<span class="badge" class:admin={u.deaf}><Icon name="headphones-off" size={13} /></span>
|
||||
{:else if u.mute || u.selfMute || u.suppress}
|
||||
<span class="badge" class:admin={u.mute || u.suppress}><Icon name="mic-off" size={13} /></span>
|
||||
{/if}
|
||||
</button>
|
||||
</li>
|
||||
{/each}
|
||||
</ul>
|
||||
{@render tiles()}
|
||||
{#if watches.some(w => w.state === 'failed')}
|
||||
<p class="failed">Could not connect to the stream. Across the internet both of you need a STUN server, chosen in Settings, Voice (the person sharing has to restart the stream afterwards).</p>
|
||||
{/if}
|
||||
{/if}
|
||||
</section>
|
||||
{/if}
|
||||
|
||||
<style>
|
||||
.stage { flex: none; border-bottom: 1px solid var(--line); background: var(--bg-1); }
|
||||
header { display: flex; align-items: center; gap: 2px; padding: 4px 8px 4px 4px; }
|
||||
.stage { position: relative; flex: none; display: flex; flex-direction: column; min-height: 0; border-bottom: 1px solid var(--line); background: var(--bg-0); }
|
||||
header { flex: none; display: flex; align-items: center; gap: 2px; padding: 4px 8px 4px 4px; background: var(--bg-1); }
|
||||
.title { flex: 1; min-width: 0; display: flex; align-items: center; gap: 8px; padding: 6px 8px; text-align: left; }
|
||||
.title:hover { background: var(--bg-hover); }
|
||||
.caret { flex: none; color: var(--text-faint); }
|
||||
@@ -79,14 +413,59 @@
|
||||
.name { font-weight: 600; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
.count { font-size: 12px; color: var(--text-faint); }
|
||||
.mini { display: flex; gap: 3px; min-width: 0; overflow: hidden; margin-left: 4px; }
|
||||
.tiles { list-style: none; margin: 0; padding: 0 10px 10px; display: grid; grid-template-columns: repeat(auto-fill, minmax(84px, 1fr)); gap: 6px;
|
||||
max-height: 40vh; overflow-y: auto; }
|
||||
.tile { position: relative; width: 100%; display: flex; flex-direction: column; align-items: center; gap: 6px; padding: 10px 4px 8px;
|
||||
background: var(--bg-2); border: 1px solid var(--line); transition: border-color 80ms, box-shadow 80ms; }
|
||||
|
||||
/* Tile view: equal 16:9 tiles, centered, sized by the script to fill the stage */
|
||||
.tiles { flex: 1; min-height: 0; padding: 8px; display: flex; flex-direction: column; gap: 8px; overflow: hidden; }
|
||||
.row { flex: 1 1 0; min-height: 0; display: flex; gap: 8px; }
|
||||
.tile { position: relative; flex: 1 1 0; min-width: 0; min-height: 0; display: flex; align-items: center; justify-content: center; overflow: hidden;
|
||||
background: var(--bg-2); border: 2px solid transparent; transition: border-color 80ms; }
|
||||
.tile:hover { background: var(--bg-hover); }
|
||||
.tile.talking { border-color: var(--speaking); box-shadow: inset 0 0 0 1px var(--speaking); }
|
||||
.who { max-width: 100%; font-size: 12px; color: var(--text-dim); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
.tile.self .who { color: var(--text); font-weight: 600; }
|
||||
.badge { position: absolute; top: 4px; right: 4px; display: inline-flex; color: var(--text-faint); }
|
||||
.tile.talking { border-color: var(--speaking); box-shadow: 0 0 0 1px var(--speaking), 0 0 18px -4px var(--speaking); }
|
||||
.tile.focused { border-color: var(--accent); }
|
||||
/* A camera keeps its own shape inside the tile, whatever size the tile is: nothing is cut
|
||||
off, a phone held upright gets bars at the sides. Your own is mirrored, like a mirror. */
|
||||
.tile video { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: contain; background: #000; }
|
||||
.tile video.mirror { transform: scaleX(-1); }
|
||||
.icon-btn.live, .icon-btn.live:hover { color: var(--speaking); }
|
||||
.camgrid { position: relative; display: flex; flex-direction: column; flex: none; }
|
||||
/* Name tag over the lower left corner */
|
||||
.who { position: absolute; left: 4px; bottom: 4px; max-width: calc(100% - 8px); padding: 1px 6px; font-size: 12px; color: #fff; background: rgb(0 0 0 / 0.6);
|
||||
white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
.tile.self .who { font-weight: 600; }
|
||||
.note { position: absolute; left: 0; right: 0; top: 4px; text-align: center; font-size: 11px; color: var(--text-dim); }
|
||||
.badge { position: absolute; top: 4px; right: 4px; display: inline-flex; padding: 2px; color: #fff; background: rgb(0 0 0 / 0.6); }
|
||||
.badge.admin { color: var(--danger); }
|
||||
.badge.live { color: var(--speaking); }
|
||||
.failed { flex: none; margin: 0; padding: 0 12px 8px; font-size: 12px; color: var(--danger); }
|
||||
|
||||
.bar { flex: none; display: flex; align-items: center; gap: 8px; padding: 6px 12px; font-size: 13px; background: var(--bg-1); border-bottom: 1px solid var(--line); }
|
||||
.bar span { flex: 1; min-width: 0; }
|
||||
.bar.bad { color: var(--danger); }
|
||||
.btn.small, .bar .btn { padding: 4px 10px; font-size: 12px; flex: none; }
|
||||
.status { display: inline-flex; align-items: center; gap: 6px; font-size: 12px; color: var(--speaking); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; min-width: 0; margin-right: 4px; }
|
||||
|
||||
/* Meeting view: the chosen picture large, everyone in one row below */
|
||||
.spot { flex: 1; min-height: 0; display: flex; flex-direction: column; background: #000; }
|
||||
.spot-head { flex: none; display: flex; align-items: center; gap: 8px; padding: 2px 6px 2px 12px; background: var(--bg-0); color: var(--text-dim); font-size: 13px; }
|
||||
.spot-who { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--text); font-weight: 600; }
|
||||
.pic { position: relative; flex: 1; min-height: 0; display: flex; -webkit-touch-callout: none; user-select: none; }
|
||||
.pending { position: absolute; inset: 0; margin: 0; display: flex; align-items: center; justify-content: center; text-align: center; padding: 16px;
|
||||
color: #ddd; font-size: 13px; background: rgb(0 0 0 / 0.6); }
|
||||
.pending.bad { color: #ffb3ad; }
|
||||
.spot video { flex: 1; min-width: 0; min-height: 0; width: 100%; height: 100%; object-fit: contain; background: #000; display: block; }
|
||||
.away { flex: 1; margin: 0; display: flex; align-items: center; justify-content: center; color: #aaa; font-size: 13px; }
|
||||
.vol { width: 110px; flex: none; padding: 0; }
|
||||
/* Video only: the picture as wide as the chat, as tall as its shape asks */
|
||||
/* It gives way when the window is short, so the chat below keeps its last lines in view */
|
||||
.stage.solo { flex: 0 1 auto; background: #000; }
|
||||
.stage.solo .spot { flex: 0 1 auto; }
|
||||
.stage.solo .pic { flex: 0 1 auto; flex-direction: column; }
|
||||
.stage.solo .spot video { flex: 0 1 auto; height: auto; max-height: 62vh; }
|
||||
.stage.solo .away { flex: none; padding: 24px; }
|
||||
.wait { margin: 0; display: flex; align-items: center; gap: 8px; padding: 8px 12px; font-size: 13px; color: var(--text-dim); background: var(--bg-1); }
|
||||
.wait span { flex: 1; min-width: 0; }
|
||||
.wait.bad { color: var(--danger); }
|
||||
/* One row of a fixed height; tiles share the width and scroll sideways when there are many */
|
||||
.tiles.strip { flex: none; height: 92px; padding: 6px 8px; overflow-x: auto; overflow-y: hidden; }
|
||||
.tiles.strip .tile { flex: 1 0 120px; max-width: 220px; }
|
||||
</style>
|
||||
|
||||
+53
-1
@@ -226,6 +226,16 @@ try {
|
||||
assert.equal(await page.evaluate(() => (window as any).pwned), undefined);
|
||||
assert.equal(await page.locator('.msg b', { hasText: 'bold' }).count(), 1);
|
||||
assert.equal(await page.locator('img[src^="https://evil"]').count(), 0);
|
||||
// The full date appears at the right edge while the pointer is on a message
|
||||
const anyMsg = page.locator('.chat .msg').last();
|
||||
const stamp = anyMsg.locator('.stamp');
|
||||
assert.equal(await stamp.evaluate(el => getComputedStyle(el).visibility), 'hidden');
|
||||
await anyMsg.hover();
|
||||
assert.equal(await stamp.evaluate(el => getComputedStyle(el).visibility), 'visible');
|
||||
assert.match(await stamp.innerText(), /\d{4}.*\d{1,2}:\d{2}:\d{2}/);
|
||||
const gap = await anyMsg.evaluate(el => el.getBoundingClientRect().right - el.querySelector('.stamp')!.getBoundingClientRect().right);
|
||||
assert.ok(gap >= 0 && gap < 20, `stamp at the right edge: ${gap}`);
|
||||
await page.mouse.move(5, 5);
|
||||
console.log('ok: incoming text rendered and sanitized');
|
||||
|
||||
// Right-click menus for text: composer editing, links, messages, authors
|
||||
@@ -324,6 +334,21 @@ try {
|
||||
await page.getByRole('button', { name: 'Done', exact: true }).click();
|
||||
console.log('ok: quick audio options on right-click');
|
||||
|
||||
// STUN server for screen sharing: none by default, presets, or a typed address
|
||||
await page.getByTitle('Settings').click();
|
||||
const stun = page.getByLabel('STUN server', { exact: true });
|
||||
assert.equal(await stun.inputValue(), '');
|
||||
await stun.selectOption({ label: 'Cloudflare (stun.cloudflare.com:3478)' });
|
||||
assert.equal(await page.evaluate(() => JSON.parse(localStorage.getItem('mumh5.settings')!).stunServer), 'stun.cloudflare.com:3478');
|
||||
await stun.selectOption('custom');
|
||||
await page.getByLabel('STUN server address').fill('stun.example.org:3478');
|
||||
await page.getByLabel('STUN server address').blur();
|
||||
assert.equal(await page.evaluate(() => JSON.parse(localStorage.getItem('mumh5.settings')!).stunServer), 'stun.example.org:3478');
|
||||
await stun.selectOption('');
|
||||
assert.equal(await page.evaluate(() => JSON.parse(localStorage.getItem('mumh5.settings')!).stunServer), '');
|
||||
await page.getByRole('button', { name: 'Done', exact: true }).click();
|
||||
console.log('ok: STUN server choices');
|
||||
|
||||
// Color schemes apply right away and are remembered
|
||||
await page.getByTitle('Settings').click();
|
||||
await page.getByRole('tab', { name: 'Appearance' }).click();
|
||||
@@ -706,7 +731,15 @@ try {
|
||||
await page.locator('.channel .name', { hasText: 'Lobby' }).click({ button: 'right' });
|
||||
await page.getByRole('menuitem', { name: 'Join channel' }).click();
|
||||
await page.locator('.me .who span', { hasText: 'Lobby' }).waitFor();
|
||||
console.log('ok: channel context menu joins');
|
||||
// Messaging a channel from its menu opens the side chat, ready to type, without joining
|
||||
await page.locator('.channel .name', { hasText: 'Games' }).click({ button: 'right' });
|
||||
await page.getByRole('menuitem', { name: 'Send message to channel' }).click();
|
||||
const sideInput = page.locator('.panel .composer [contenteditable]');
|
||||
await sideInput.waitFor();
|
||||
assert.equal(await sideInput.getAttribute('data-placeholder'), 'Message #Games');
|
||||
await page.waitForFunction(() => document.activeElement === document.querySelector('.panel .composer [contenteditable]'), null, { timeout: 3000 });
|
||||
await page.locator('.me .who span', { hasText: 'Lobby' }).waitFor();
|
||||
console.log('ok: channel context menu joins, and messages a channel from the side chat');
|
||||
|
||||
// Side panel can be resized by dragging its left edge
|
||||
const panelBox = (await page.locator('aside.panel').boundingBox())!;
|
||||
@@ -846,6 +879,19 @@ try {
|
||||
assert.ok(inline.length <= 131072, `inline image within the server limit (${inline.length})`);
|
||||
console.log(`ok: inline image without upload host (${inline.length} chars)`);
|
||||
|
||||
// A click on an image in the chat shows it large; Escape or a click closes it
|
||||
await page.locator('.chat .msg .html img').last().click();
|
||||
const viewer = page.getByRole('dialog', { name: 'Image' });
|
||||
await viewer.waitFor();
|
||||
const shown = await viewer.locator('img').evaluate((i: HTMLImageElement) => [i.naturalWidth > 0, i.getBoundingClientRect().width]);
|
||||
assert.ok(shown[0] && (shown[1] as number) > 300, `image shown large: ${shown}`);
|
||||
await page.keyboard.press('Escape');
|
||||
await viewer.waitFor({ state: 'detached' });
|
||||
await page.locator('.chat .msg .html img').last().click();
|
||||
await viewer.getByRole('button', { name: 'Close image' }).click({ position: { x: 5, y: 5 } });
|
||||
await viewer.waitFor({ state: 'detached' });
|
||||
console.log('ok: images open large on click');
|
||||
|
||||
// ─── Several servers at once ─────────────────────────────────────────────
|
||||
const userState = (name: string) => [...bob.users.values()].find(u => u.name === name);
|
||||
const until2 = (label: string, cond: () => boolean) => within(new Promise<void>(res => {
|
||||
@@ -871,6 +917,10 @@ try {
|
||||
await page.locator('.channel .name', { hasText: 'Root' }).dblclick();
|
||||
await until2('voice moved to the second server', () => !userState('alice-b')?.selfDeaf && !!userState('alice')?.selfDeaf);
|
||||
assert.equal(await page.locator('.rail .tile.voice').getAttribute('title').then(t => t?.startsWith('Second')), true);
|
||||
// The server with voice leads the rail, although it was added last
|
||||
const railTop = (title: string) => page.locator(`.rail .tile[title^="${title}"]`).boundingBox().then(b => b!.y);
|
||||
assert.ok(await railTop('Second') < await railTop('Test Server'), 'voice server on top');
|
||||
assert.match(await page.locator('.rail .tile').first().getAttribute('title') ?? '', /^Test Server/, 'the saved order is untouched');
|
||||
console.log('ok: voice follows the channel you join');
|
||||
|
||||
// Messages on the server not on screen show as unread on its tile
|
||||
@@ -886,6 +936,8 @@ try {
|
||||
// Custom icon from the tile's context menu: square-cropped and shown instead of initials
|
||||
const iconFile = path.join(userData, 'icon.png');
|
||||
writeFileSync(iconFile, testPng(300, 200));
|
||||
// With hints off, the title of whatever the pointer rests on is set aside; move off the tile first
|
||||
await page.mouse.move(700, 400);
|
||||
await page.locator('.rail .tile[title^="Second"]').click({ button: 'right' });
|
||||
const chooser = page.waitForEvent('filechooser');
|
||||
await page.getByRole('menuitem', { name: 'Change icon...' }).click();
|
||||
|
||||
@@ -0,0 +1,385 @@
|
||||
// Screen sharing between two instances of the built app on a real Mumble server.
|
||||
// npm run build && MUMBLE_TEST_HOST=localhost:64739 npm run test:e2e:share
|
||||
import { _electron as electron, type Page } from 'playwright-core';
|
||||
import electronPath from 'electron';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtempSync } from 'node:fs';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
const target = process.env.MUMBLE_TEST_HOST;
|
||||
if (!target) {
|
||||
console.log('MUMBLE_TEST_HOST not set, skipping');
|
||||
process.exit(0);
|
||||
}
|
||||
const [host, port] = target.split(':');
|
||||
const root = path.resolve(import.meta.dirname, '../..');
|
||||
const { ELECTRON_RUN_AS_NODE, ...env } = process.env;
|
||||
const tag = Date.now() % 100000;
|
||||
|
||||
async function launch() {
|
||||
const app = await electron.launch({
|
||||
executablePath: electronPath as unknown as string,
|
||||
args: [root, `--user-data-dir=${mkdtempSync(path.join(tmpdir(), 'mumh5-e2e-'))}`, '--ozone-platform=x11',
|
||||
'--use-fake-device-for-media-stream=device-count=2', '--use-fake-ui-for-media-stream'],
|
||||
env: env as Record<string, string>
|
||||
});
|
||||
const page = await app.firstWindow();
|
||||
page.on('console', m => { if (m.type() === 'error') console.log('[renderer]', m.text()); });
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
return { app, page };
|
||||
}
|
||||
|
||||
// Identity wizard, then connect; the username comes from the identity
|
||||
async function join(page: Page, name: string) {
|
||||
await page.getByRole('dialog', { name: 'Set up your identity' }).waitFor();
|
||||
await page.getByRole('button', { name: /Create a new identity/ }).click();
|
||||
await page.getByLabel('Name', { exact: true }).fill(name);
|
||||
await page.getByRole('button', { name: 'Create', exact: true }).click();
|
||||
await page.getByRole('button', { name: 'Skip for now' }).click();
|
||||
await page.getByRole('button', { name: 'Done' }).click();
|
||||
await page.getByTitle('Add a server').click();
|
||||
await page.getByLabel('Address').fill(host);
|
||||
await page.getByLabel('Port').fill(port);
|
||||
await page.getByRole('button', { name: 'Save and connect' }).click();
|
||||
await page.getByText(/Connected/).first().waitFor();
|
||||
}
|
||||
|
||||
const cleanup: (() => void)[] = [];
|
||||
const alice = await launch();
|
||||
const bob = await launch();
|
||||
const aliceName = `share-a-${tag}`, bobName = `share-b-${tag}`;
|
||||
try {
|
||||
await join(alice.page, aliceName);
|
||||
|
||||
// Alice starts sharing: a note about IP addresses once, then the source picker
|
||||
const shareBtn = alice.page.locator('.me').getByRole('button', { name: 'Share your screen' });
|
||||
await shareBtn.click();
|
||||
await alice.page.getByText(/You see each other's IP address/).waitFor();
|
||||
await alice.page.getByRole('button', { name: 'Continue' }).click();
|
||||
// The dialog: pick a source, see the preview, then start
|
||||
const picker = alice.page.getByRole('dialog', { name: 'Share your screen' });
|
||||
const startBtn = picker.getByRole('button', { name: 'Start sharing' });
|
||||
const pickAndStart = async () => {
|
||||
assert.ok(await startBtn.isDisabled(), 'nothing can start before a source is chosen');
|
||||
// Listing is slow on the bare test display and can come back empty the first time
|
||||
for (let i = 0; i < 6 && !(await picker.locator('.sources button').count()); i++) {
|
||||
await Promise.race([picker.locator('.sources button').first().waitFor({ timeout: 40000 }), picker.getByText('Nothing found').waitFor({ timeout: 40000 })]).catch(() => {});
|
||||
if (await picker.getByText('Nothing found').count()) await picker.getByRole('button', { name: 'Refresh' }).click();
|
||||
}
|
||||
await picker.locator('.sources button').first().click();
|
||||
await alice.page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 });
|
||||
await startBtn.click();
|
||||
};
|
||||
await picker.getByLabel('Quality').selectOption('low');
|
||||
await pickAndStart();
|
||||
await alice.page.locator('.stage .status', { hasText: '0 watching' }).waitFor();
|
||||
await alice.page.locator('.sidebar .row.user.self').getByTitle('You are sharing your screen').waitFor();
|
||||
// The sharer sees what is being sent, large by default; the tiles themselves never carry the picture
|
||||
const ownView = alice.page.getByRole('region', { name: 'Screen share' });
|
||||
await ownView.getByText('Your screen').waitFor();
|
||||
await alice.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 });
|
||||
assert.equal(await alice.page.locator('.stage .tile video').count(), 0);
|
||||
await ownView.getByRole('button', { name: 'Back to tiles' }).click();
|
||||
await ownView.waitFor({ state: 'detached' });
|
||||
await alice.page.locator('.stage .tile.self').click();
|
||||
await ownView.waitFor();
|
||||
console.log('ok: sharing started from the dialog, own stream shown large');
|
||||
|
||||
// Stacked layout: only the picture, as wide as the chat, and the layout stays stacked on narrow windows
|
||||
const setLayout = async (name: string) => {
|
||||
await alice.page.getByTitle('Settings').click();
|
||||
await alice.page.getByRole('tab', { name: 'Appearance' }).click();
|
||||
await alice.page.getByRole('radio', { name }).click();
|
||||
await alice.page.getByRole('button', { name: 'Done', exact: true }).click();
|
||||
};
|
||||
await setLayout('Stacked');
|
||||
await alice.page.locator('.app.stacked .stage.solo').waitFor();
|
||||
assert.equal(await alice.page.locator('.stage .tiles').count(), 0, 'no participant tiles in the stacked layout');
|
||||
const fits = () => alice.page.evaluate(() => {
|
||||
const v = document.querySelector('.stage.solo video') as HTMLVideoElement, c = document.querySelector('.chat')!;
|
||||
return v.videoWidth > 0 && Math.abs(v.getBoundingClientRect().width - c.getBoundingClientRect().width) < 2;
|
||||
});
|
||||
await alice.page.waitForFunction(() => (document.querySelector('.stage.solo video') as HTMLVideoElement | null)?.videoWidth! > 0);
|
||||
assert.ok(await fits(), 'picture as wide as the chat');
|
||||
await alice.page.setViewportSize({ width: 600, height: 800 });
|
||||
await alice.page.locator('.app.stacked .stage.solo video').waitFor();
|
||||
assert.ok(await fits(), 'still as wide as the chat on a narrow window');
|
||||
// On a short window the picture gives way and the chat keeps about five lines
|
||||
await alice.page.setViewportSize({ width: 600, height: 700 });
|
||||
const heights = () => alice.page.evaluate(() => [document.querySelector('.chat [role=log]')!.getBoundingClientRect().height, document.querySelector('.stage.solo video')!.getBoundingClientRect().height]);
|
||||
await alice.page.waitForFunction(() => document.querySelector('.stage.solo video')!.getBoundingClientRect().height < 250);
|
||||
const [logH, videoH] = await heights();
|
||||
assert.ok(logH >= 120, `chat keeps its lines: ${logH}`);
|
||||
assert.ok(videoH > 40, `the picture is still shown: ${videoH}`);
|
||||
await alice.page.setViewportSize({ width: 600, height: 800 });
|
||||
// The chat can be dragged taller, never below its minimum; the picture takes the rest
|
||||
const chatHandle = alice.page.getByRole('separator', { name: 'Resize the chat' });
|
||||
await chatHandle.focus();
|
||||
for (let i = 0; i < 12; i++) await alice.page.keyboard.press('ArrowUp');
|
||||
await alice.page.waitForFunction(() => document.querySelector('.chat [role=log]')!.getBoundingClientRect().height > 200);
|
||||
const grown = (await heights())[0];
|
||||
for (let i = 0; i < 40; i++) await alice.page.keyboard.press('ArrowDown');
|
||||
await alice.page.waitForFunction(() => document.querySelector('.chat [role=log]')!.getBoundingClientRect().height < 130);
|
||||
const [smallest, videoNow] = await heights();
|
||||
assert.ok(smallest >= 124, `not below the minimum: ${smallest}`);
|
||||
assert.ok(videoNow > 100, `the picture fills the rest: ${videoNow}`);
|
||||
console.log(`ok: stacked chat resizable (${Math.round(grown)} px down to ${Math.round(smallest)} px)`);
|
||||
if (process.env.SHOTS_DIR) await alice.page.screenshot({ path: `${process.env.SHOTS_DIR}/stacked.png` });
|
||||
await alice.page.getByText('0 watching').waitFor();
|
||||
await alice.page.setViewportSize({ width: 1280, height: 800 });
|
||||
await setLayout('Modern');
|
||||
await alice.page.locator('.stage .tiles').waitFor();
|
||||
console.log('ok: stacked layout shows only the picture and stays stacked when narrow');
|
||||
|
||||
// Bob connects afterwards and still learns about the stream
|
||||
await join(bob.page, bobName);
|
||||
// Next to the name in the channel list, and on the tile above the chat
|
||||
const watch = bob.page.locator('.sidebar').getByRole('button', { name: `Watch the screen of ${aliceName}`, exact: true });
|
||||
await watch.waitFor();
|
||||
const tile = bob.page.locator('.stage .tile', { hasText: aliceName });
|
||||
await tile.waitFor();
|
||||
console.log('ok: a late joiner sees who is sharing');
|
||||
|
||||
// The tiles fill the stage exactly, whatever its size
|
||||
const fills = () => bob.page.evaluate(() => {
|
||||
const grid = document.querySelector('.stage .tiles')!.getBoundingClientRect();
|
||||
const tiles = [...document.querySelectorAll('.stage .tile')].map(e => e.getBoundingClientRect());
|
||||
const left = Math.min(...tiles.map(r => r.left)), right = Math.max(...tiles.map(r => r.right));
|
||||
const top = Math.min(...tiles.map(r => r.top)), bottom = Math.max(...tiles.map(r => r.bottom));
|
||||
return { ok: left - grid.left < 10 && grid.right - right < 10 && top - grid.top < 10 && grid.bottom - bottom < 10, height: Math.round(tiles[0].height), count: tiles.length };
|
||||
});
|
||||
const first = await fills();
|
||||
assert.ok(first.ok && first.count === 2, `tiles fill the stage: ${JSON.stringify(first)}`);
|
||||
const handle = bob.page.getByRole('separator', { name: 'Resize the tiles' });
|
||||
await handle.focus();
|
||||
for (let i = 0; i < 8; i++) await bob.page.keyboard.press('ArrowDown');
|
||||
await bob.page.waitForFunction(h => Math.round(document.querySelector('.stage .tile')!.getBoundingClientRect().height) > h, first.height);
|
||||
const taller = await fills();
|
||||
assert.ok(taller.ok, 'still filled after making the stage taller');
|
||||
await bob.page.setViewportSize({ width: 820, height: 700 });
|
||||
await bob.page.waitForFunction(() => document.querySelector('.stage .tiles')!.getBoundingClientRect().width < 700);
|
||||
assert.ok((await fills()).ok, 'still filled after making the window narrower');
|
||||
await bob.page.setViewportSize({ width: 1280, height: 800 });
|
||||
for (let i = 0; i < 8; i++) await bob.page.keyboard.press('ArrowUp');
|
||||
if (process.env.SHOTS_DIR) await bob.page.screenshot({ path: `${process.env.SHOTS_DIR}/tiles.png` });
|
||||
console.log(`ok: tiles fill the stage at every size (${first.height} to ${taller.height} px tall)`);
|
||||
|
||||
await tile.click();
|
||||
await bob.page.getByRole('button', { name: 'Continue' }).click();
|
||||
const view = bob.page.getByRole('region', { name: 'Screen share' });
|
||||
await view.getByText(aliceName).waitFor();
|
||||
await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 });
|
||||
const size = await bob.page.evaluate(() => { const v = document.querySelector('.spot video') as HTMLVideoElement; return [v.videoWidth, v.videoHeight]; });
|
||||
await alice.page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
|
||||
console.log(`ok: bob receives the picture (${size[0]}x${size[1]}), alice sees one viewer`);
|
||||
|
||||
if (process.env.SHOTS_DIR) { await bob.page.waitForTimeout(500); await bob.page.screenshot({ path: `${process.env.SHOTS_DIR}/meeting.png` }); }
|
||||
// Right-click on the picture: statistics for both sides
|
||||
const nerd = async (page: Page, expect: RegExp[]) => {
|
||||
await page.locator('.spot video').click({ button: 'right' });
|
||||
await page.getByRole('menuitem', { name: 'Stats for nerds' }).click();
|
||||
const box = page.getByRole('status', { name: 'Stream statistics' });
|
||||
for (const e of expect) await box.getByText(e).first().waitFor({ timeout: 8000 });
|
||||
const text = (await box.innerText()).replace(/\n+/g, ' | ');
|
||||
await box.getByRole('button', { name: 'Close statistics' }).click();
|
||||
await box.waitFor({ state: 'detached' });
|
||||
return text;
|
||||
};
|
||||
const viewerStats = await nerd(bob.page, [/^(VP8|VP9|H264|AV1)/, /^\d+x\d+/, /^\d+ kbit\/s$/, /^direct/]);
|
||||
const sharerStats = await nerd(alice.page, [new RegExp(`To ${bobName}`), /^(VP8|VP9|H264|AV1)/, /^\d+ kbit\/s$/]);
|
||||
console.log(`ok: stats for nerds (viewer: ${viewerStats.slice(0, 200)}...)`);
|
||||
assert.ok(sharerStats.includes('Round trip'));
|
||||
|
||||
// Touch screens: holding a finger on the picture opens the same menu
|
||||
await bob.page.evaluate(() => {
|
||||
const el = document.querySelector('.spot .pic')!;
|
||||
const r = el.getBoundingClientRect();
|
||||
const touch = new Touch({ identifier: 1, target: el, clientX: r.left + 40, clientY: r.top + 40 });
|
||||
el.dispatchEvent(new TouchEvent('touchstart', { touches: [touch], bubbles: true }));
|
||||
});
|
||||
await bob.page.getByRole('menuitem', { name: 'Stats for nerds' }).waitFor({ timeout: 3000 });
|
||||
await bob.page.keyboard.press('Escape');
|
||||
console.log('ok: long press on the picture opens the menu');
|
||||
|
||||
// The large picture can move to a window of its own
|
||||
const opened = bob.app.waitForEvent('window');
|
||||
// From the right-click menu on the picture
|
||||
await bob.page.locator('.spot video').click({ button: 'right' });
|
||||
await bob.page.getByRole('menuitem', { name: 'Open in a separate window' }).click();
|
||||
const streamWindow = await opened;
|
||||
await streamWindow.waitForFunction(() => (document.querySelector('video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 });
|
||||
await view.getByText('Shown in a separate window.').waitFor();
|
||||
await streamWindow.close();
|
||||
await view.getByRole('button', { name: 'Open in a separate window' }).waitFor();
|
||||
await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0);
|
||||
console.log('ok: stream in a separate window, back when it is closed');
|
||||
|
||||
// Back to tiles hides the picture; the sharer's tile brings it back
|
||||
await view.getByRole('button', { name: 'Back to tiles' }).click();
|
||||
await view.waitFor({ state: 'detached' });
|
||||
assert.equal(await bob.page.locator('.stage .tile video').count(), 0);
|
||||
await tile.click();
|
||||
await view.waitFor();
|
||||
console.log('ok: large view hidden and shown again from the tile');
|
||||
|
||||
// Leaving and coming back
|
||||
await view.getByRole('button', { name: 'Stop watching' }).click();
|
||||
await view.waitFor({ state: 'detached' });
|
||||
await alice.page.locator('.stage .status', { hasText: '0 watching' }).waitFor();
|
||||
// The sharer's profile says so and offers to watch
|
||||
await bob.page.locator('.sidebar .row.user', { hasText: aliceName }).click();
|
||||
const profile = bob.page.locator('.panel .sharing');
|
||||
await profile.getByText('Sharing their screen').waitFor();
|
||||
await profile.getByRole('button', { name: 'Watch' }).click();
|
||||
await profile.getByRole('button', { name: 'Stop watching' }).waitFor();
|
||||
await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 });
|
||||
console.log('ok: stop watching, watch again from the profile');
|
||||
|
||||
// Alice stops: the view and the indicator go away
|
||||
await alice.page.locator('.me').getByRole('button', { name: 'Stop sharing your screen' }).click();
|
||||
await view.waitFor({ state: 'detached' });
|
||||
await watch.waitFor({ state: 'detached' });
|
||||
await profile.waitFor({ state: 'detached' });
|
||||
assert.equal(await alice.page.locator('.stage .status').count(), 0);
|
||||
console.log('ok: stopping ends the stream for viewers');
|
||||
|
||||
// Camera instead of the screen, and a relay for networks without direct connections: both
|
||||
// instances point at a mumh5 proxy, and the viewer is limited to the relay over TCP
|
||||
const relayPort = 19000 + Math.floor(Math.random() * 500);
|
||||
const relay = spawn(process.execPath, ['server/main.ts'], {
|
||||
cwd: root, stdio: ['ignore', 'pipe', 'inherit'],
|
||||
env: { ...env, MUMH5_PORT: String(relayPort), MUMH5_ALLOW_ANY: '1', MUMH5_ALLOW_PRIVATE: '1', MUMH5_STUN_PORT: String(relayPort + 500), MUMH5_STUN_BIND: '127.0.0.1', MUMH5_TURN_IP: '127.0.0.1' }
|
||||
});
|
||||
cleanup.push(() => relay.kill());
|
||||
await new Promise<void>((res, rej) => { relay.stdout.on('data', d => { if (String(d).includes('listening')) res(); }); relay.on('exit', c => rej(new Error(`proxy exited with ${c}`))); });
|
||||
for (const p of [alice.page, bob.page]) {
|
||||
await p.getByTitle('Settings').click();
|
||||
await p.getByLabel('Relay').fill(`http://127.0.0.1:${relayPort}`);
|
||||
await p.getByLabel('Relay').blur();
|
||||
await p.getByRole('button', { name: 'Done', exact: true }).click();
|
||||
}
|
||||
await bob.page.evaluate(() => localStorage.setItem('mumh5.iceDebug', 'relay-tcp'));
|
||||
|
||||
await shareBtn.click();
|
||||
await picker.getByRole('tab', { name: 'Camera' }).click();
|
||||
await alice.page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 });
|
||||
assert.equal(await picker.getByLabel('Sound').count(), 0, 'no extra sound with a camera');
|
||||
await startBtn.click();
|
||||
await alice.page.locator('.sidebar .row.user.self').getByTitle('You are sharing your camera').waitFor();
|
||||
// Cameras play in the tiles and come on by themselves, like in a video call
|
||||
const cam = bob.page.locator('.sidebar').getByRole('button', { name: `Watch the camera of ${aliceName}`, exact: true });
|
||||
await cam.waitFor();
|
||||
const liveCams = (p: Page) => p.evaluate(() => [...document.querySelectorAll('.stage .tile video')].filter(v => (v as HTMLVideoElement).videoWidth > 0).length);
|
||||
const camsShown = (p: Page, n: number) => p.waitForFunction(
|
||||
want => [...document.querySelectorAll('.stage .tile video')].filter(v => (v as HTMLVideoElement).videoWidth > 0).length === want, n, { timeout: 30000 });
|
||||
await camsShown(bob.page, 1);
|
||||
// One person's camera can be hidden, and shown again
|
||||
await bob.page.locator('.stage .tile', { hasText: aliceName }).click({ button: 'right' });
|
||||
await bob.page.getByRole('menuitem', { name: 'Hide camera' }).click();
|
||||
await camsShown(bob.page, 0);
|
||||
await alice.page.locator('.stage .status', { hasText: '0 watching' }).waitFor();
|
||||
await bob.page.locator('.stage .tile', { hasText: aliceName }).click({ button: 'right' });
|
||||
await bob.page.getByRole('menuitem', { name: 'Show camera' }).click();
|
||||
await camsShown(bob.page, 1);
|
||||
assert.equal(await bob.page.getByRole('region', { name: 'Screen share' }).count(), 0, 'a camera does not take over the stage by itself');
|
||||
|
||||
// Bob turns his camera on too: both now see both, each one their own mirrored
|
||||
await bob.page.locator('.me').getByRole('button', { name: 'Share your screen or camera' }).click();
|
||||
const bobDialog = bob.page.getByRole('dialog', { name: 'Share your screen' });
|
||||
await bobDialog.getByRole('tab', { name: 'Camera' }).click();
|
||||
await bob.page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 });
|
||||
await bobDialog.getByRole('button', { name: 'Start sharing' }).click();
|
||||
await camsShown(bob.page, 2);
|
||||
await camsShown(alice.page, 2);
|
||||
assert.equal(await bob.page.locator('.stage .tile.self video.mirror').count(), 1, 'own camera is mirrored');
|
||||
// The picture keeps its shape in the tile at any tile size: whole, not cropped
|
||||
const shape = () => bob.page.evaluate(() => [...document.querySelectorAll('.stage .tile video')].map(v => {
|
||||
const el = v as HTMLVideoElement, r = el.getBoundingClientRect();
|
||||
const scale = Math.min(r.width / el.videoWidth, r.height / el.videoHeight);
|
||||
return { fit: getComputedStyle(el).objectFit, inside: el.videoWidth * scale <= r.width + 1 && el.videoHeight * scale <= r.height + 1, box: r.width > 50 && r.height > 50 };
|
||||
}));
|
||||
for (const size of [{ width: 1280, height: 800 }, { width: 760, height: 620 }]) {
|
||||
await bob.page.setViewportSize(size);
|
||||
await camsShown(bob.page, 2);
|
||||
assert.ok((await shape()).every(s => s.fit === 'contain' && s.inside && s.box), `cameras whole at ${size.width} wide`);
|
||||
}
|
||||
await bob.page.setViewportSize({ width: 1280, height: 800 });
|
||||
assert.equal(await bob.page.locator('.stage .tile:not(.self) video.mirror').count(), 0);
|
||||
await alice.page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
|
||||
await bob.page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
|
||||
if (process.env.SHOTS_DIR) await bob.page.screenshot({ path: `${process.env.SHOTS_DIR}/cams.png` });
|
||||
|
||||
// Switching cameras swaps the picture on the running stream. Chromium's second fake camera
|
||||
// never delivers frames to an encoder (not even as the first camera), so the viewer is
|
||||
// checked after switching there and back.
|
||||
const ownCamera = () => alice.page.evaluate(() => ((document.querySelector('.stage .tile.self video') as HTMLVideoElement).srcObject as MediaStream).getVideoTracks()[0].label);
|
||||
const firstCamera = await ownCamera();
|
||||
const switchTo = async (different: boolean) => {
|
||||
await alice.page.locator('.stage').getByRole('button', { name: 'Switch camera' }).click();
|
||||
await alice.page.waitForFunction(([was, differs]) => {
|
||||
const v = document.querySelector('.stage .tile.self video') as HTMLVideoElement | null;
|
||||
const track = (v?.srcObject as MediaStream | null)?.getVideoTracks()[0];
|
||||
return !!track && (track.label !== was) === differs && track.readyState === 'live' && v!.videoWidth > 0;
|
||||
}, [firstCamera, different] as const, { timeout: 15000 });
|
||||
await alice.page.getByRole('button', { name: 'Switch camera' }).and(alice.page.locator(':enabled')).waitFor();
|
||||
};
|
||||
await switchTo(true);
|
||||
const secondCamera = await ownCamera();
|
||||
await switchTo(false);
|
||||
const fresh = await bob.page.evaluate(n => new Promise<number>(resolve => {
|
||||
const v = document.querySelector(`.stage .tile[aria-label*="${n}"] video`) as HTMLVideoElement;
|
||||
let count = 0;
|
||||
const tick = () => { if (++count >= 8) resolve(count); else v.requestVideoFrameCallback(tick); };
|
||||
v.requestVideoFrameCallback(tick);
|
||||
setTimeout(() => resolve(count), 12000);
|
||||
}), aliceName);
|
||||
assert.ok(fresh >= 8, `viewer got ${fresh} new frames after switching cameras`);
|
||||
await alice.page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
|
||||
assert.equal(await alice.page.locator('.bar.bad').count(), 0);
|
||||
console.log(`ok: camera switched while sharing (${firstCamera} to ${secondCamera} and back), viewer kept the stream`);
|
||||
|
||||
// A click enlarges one camera; the statistics show it came through the relay
|
||||
await bob.page.locator('.stage .tile', { hasText: aliceName }).click();
|
||||
await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 });
|
||||
assert.equal(await bob.page.locator('.stage .tile', { hasText: aliceName }).locator('video').count(), 0, 'the enlarged camera is not also in its tile');
|
||||
assert.equal(await bob.page.locator('.stage .tile.self video').count(), 1, 'the other tiles keep theirs');
|
||||
const camStats = await nerd(bob.page, [/^relayed \(TURN\), reached over TCP/, /^0 direct, 0 through STUN, [1-9]\d* relayed/]);
|
||||
await bob.page.getByRole('region', { name: 'Screen share' }).getByRole('button', { name: 'Back to tiles' }).click();
|
||||
|
||||
// Everyone's cameras off at once keeps Bob's own camera on and drops the others; on again brings them back
|
||||
await bob.page.locator('.stage').getByRole('button', { name: 'Hide all cameras' }).click();
|
||||
await camsShown(bob.page, 1);
|
||||
await alice.page.locator('.stage .status', { hasText: '0 watching' }).waitFor();
|
||||
await bob.page.locator('.stage').getByRole('button', { name: 'Show all cameras' }).click();
|
||||
await camsShown(bob.page, 2);
|
||||
await bob.page.locator('.me').getByRole('button', { name: 'Stop sharing your camera' }).click();
|
||||
await camsShown(alice.page, 1);
|
||||
await alice.page.locator('.me').getByRole('button', { name: 'Stop sharing your camera' }).click();
|
||||
await cam.waitFor({ state: 'detached' });
|
||||
await bob.page.evaluate(() => localStorage.removeItem('mumh5.iceDebug'));
|
||||
console.log(`ok: cameras of several people at once, in the tiles, through the relay (${camStats.match(/Picture \| [^|]+/)?.[0]})`);
|
||||
void liveCams;
|
||||
|
||||
// A sharer who disconnects takes the stream with them
|
||||
await shareBtn.click();
|
||||
await pickAndStart();
|
||||
await watch.waitFor();
|
||||
await alice.page.getByTitle('Disconnect').click();
|
||||
await watch.waitFor({ state: 'detached' });
|
||||
console.log('ok: disconnecting ends the stream');
|
||||
|
||||
// Sounds: the sharer hears the stream start and stop and viewers come and go; the channel hears streams start and stop
|
||||
const sounds = (p: Page) => p.evaluate(() => [...((window as any).__mumh5Sounds ?? [])] as string[]);
|
||||
const heardA = await sounds(alice.page), heardB = await sounds(bob.page);
|
||||
for (const ev of ['shareStarted', 'viewerJoined', 'viewerLeft', 'shareStopped']) assert.ok(heardA.includes(ev), `sharer hears ${ev}: ${heardA}`);
|
||||
for (const ev of ['shareStarted', 'shareStopped']) assert.ok(heardB.includes(ev), `viewer hears ${ev}: ${heardB}`);
|
||||
console.log('ok: sounds for streams and viewers');
|
||||
|
||||
console.log('SHARE E2E PASSED');
|
||||
} finally {
|
||||
for (const c of cleanup) c();
|
||||
await alice.app.close().catch(() => {});
|
||||
await bob.app.close().catch(() => {});
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// A bare browser window for the web E2E: no preload, so the page runs as it would in a browser.
|
||||
const { app, BrowserWindow, session } = require('electron');
|
||||
const { app, BrowserWindow, desktopCapturer, session } = require('electron');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
@@ -11,6 +11,12 @@ app.whenReady().then(() => {
|
||||
item.setSavePath(file);
|
||||
item.once('done', (_ev, state) => { if (state === 'completed') fs.writeFileSync(file + '.done', ''); });
|
||||
});
|
||||
// Stands in for the browser's own "choose what to share" dialog: always the first screen
|
||||
session.defaultSession.setDisplayMediaRequestHandler(async (_request, callback) => {
|
||||
let sources = [];
|
||||
for (let i = 0; i < 10 && !sources.length; i++) sources = await desktopCapturer.getSources({ types: ['screen'] });
|
||||
try { callback(sources.length ? { video: sources[0] } : {}); } catch { /* refused */ }
|
||||
});
|
||||
const win = new BrowserWindow({ width: 1280, height: 800, webPreferences: { contextIsolation: true, sandbox: true, nodeIntegration: false } });
|
||||
win.loadURL(process.env.MUMH5_WEB_URL);
|
||||
});
|
||||
|
||||
+101
-3
@@ -9,7 +9,9 @@ import { mkdtempSync, existsSync, statSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { createCodec } from '../../src/core/proto.ts';
|
||||
import { decodeVoice, type VoicePacket } from '../../src/core/voice-packet.ts';
|
||||
import { decodeVoice, encodeVoice, type VoicePacket } from '../../src/core/voice-packet.ts';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { instantiateOpus, WasmDecoder } from '../../src/lib/audio/opus-wasm.ts';
|
||||
import { MumbleClient } from '../../src/core/client.ts';
|
||||
import type { Transport } from '../../src/core/transport.ts';
|
||||
import { openTls } from '../../electron/tls-transport.ts';
|
||||
@@ -42,7 +44,9 @@ function within<T>(p: Promise<T>, label: string, ms = 10000): Promise<T> {
|
||||
const proxyPort = 18000 + Math.floor(Math.random() * 1000);
|
||||
const { ELECTRON_RUN_AS_NODE, ...env } = process.env;
|
||||
const proxy = spawn(process.execPath, [path.join(root, 'dist-proxy/proxy.mjs')], {
|
||||
env: { ...env, MUMH5_PORT: String(proxyPort), MUMH5_SERVERS: `${target}=Test Server` }, stdio: ['ignore', 'pipe', 'inherit']
|
||||
env: { ...env, MUMH5_PORT: String(proxyPort), MUMH5_SERVERS: `${target}=Test Server`, MUMH5_STUN_PORT: String(proxyPort + 1000), MUMH5_STUN_BIND: '127.0.0.1',
|
||||
// Everything is on this machine, so the relay has to be allowed to reach loopback
|
||||
MUMH5_ALLOW_PRIVATE: '1', MUMH5_TURN_IP: '127.0.0.1' }, stdio: ['ignore', 'pipe', 'inherit']
|
||||
});
|
||||
await within(new Promise<void>((res, rej) => {
|
||||
proxy.stdout.on('data', d => { if (String(d).includes('listening')) res(); });
|
||||
@@ -50,12 +54,14 @@ await within(new Promise<void>((res, rej) => {
|
||||
}), 'proxy start');
|
||||
|
||||
const downloads = mkdtempSync(path.join(tmpdir(), 'mumh5-e2e-dl-'));
|
||||
const app = await electron.launch({
|
||||
const browser = () => electron.launch({
|
||||
executablePath: electronPath as unknown as string,
|
||||
args: [path.join(root, 'test/e2e/web-shell.cjs'), `--user-data-dir=${mkdtempSync(path.join(tmpdir(), 'mumh5-e2e-'))}`, '--ozone-platform=x11',
|
||||
'--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'],
|
||||
env: { ...env, MUMH5_WEB_URL: `http://127.0.0.1:${proxyPort}/`, MUMH5_DOWNLOADS: downloads } as Record<string, string>
|
||||
});
|
||||
const app = await browser();
|
||||
let second: Awaited<ReturnType<typeof browser>> | null = null;
|
||||
const bob = await headless('bob');
|
||||
const name = `webalice${Date.now() % 100000}`;
|
||||
try {
|
||||
@@ -160,9 +166,101 @@ try {
|
||||
assert.equal(hash(), before, 'same certificate after a reload');
|
||||
console.log('ok: identity kept across a reload');
|
||||
|
||||
// Screen sharing between two browsers: the proxy's own STUN is used without any setting
|
||||
assert.equal((await (await fetch(`http://127.0.0.1:${proxyPort}/api/config`)).json()).stun, proxyPort + 1000);
|
||||
second = await browser();
|
||||
const page2 = await second.firstWindow();
|
||||
page2.on('console', m => { if (m.type() === 'error') console.log('[page2]', m.text()); });
|
||||
await page2.setViewportSize({ width: 1280, height: 800 });
|
||||
await page2.getByRole('button', { name: /Create a new identity/ }).click();
|
||||
const carol = `webcarol${Date.now() % 100000}`;
|
||||
// This browser is made to use the built-in Opus codec, as one without WebCodecs audio would
|
||||
await page2.evaluate(() => localStorage.setItem('mumh5.forceWasmOpus', '1'));
|
||||
await page2.getByLabel('Name', { exact: true }).fill(carol);
|
||||
await page2.getByRole('button', { name: 'Create', exact: true }).click();
|
||||
await page2.getByRole('button', { name: 'Skip for now' }).click();
|
||||
await page2.getByRole('button', { name: 'Done' }).click();
|
||||
await page2.getByTitle('Add a server').click();
|
||||
await page2.getByRole('button', { name: 'Save and connect' }).click();
|
||||
await page2.getByText(/Connected/).first().waitFor();
|
||||
|
||||
// Talking with the built-in codec: its packets are real Opus, checked by decoding them here
|
||||
const carolSession = () => [...bob.users.values()].find(u => u.name === carol)!.session;
|
||||
const carolVoice = new Promise<VoicePacket[]>(res => {
|
||||
const packets: VoicePacket[] = [];
|
||||
const off = bob.on('voice', raw => {
|
||||
const p = decodeVoice(raw);
|
||||
if (!p || p.session !== carolSession() || !p.opus.length) return;
|
||||
packets.push(p);
|
||||
if (packets.length >= 20) { off(); res(packets); }
|
||||
});
|
||||
});
|
||||
await page2.getByTitle('Settings').click();
|
||||
await page2.getByRole('radio', { name: /Always on/ }).click();
|
||||
await page2.getByRole('button', { name: 'Done', exact: true }).click();
|
||||
const fromCarol = await within(carolVoice, 'voice from the built-in codec', 20000);
|
||||
const opus = await instantiateOpus(readFileSync(path.join(root, 'node_modules/@evan/opus/wasm/opus.wasm')));
|
||||
const check = new WasmDecoder(opus);
|
||||
const decoded = fromCarol.map(p => check.decode(p.opus));
|
||||
assert.ok(decoded.every(pcm => pcm && pcm.length === 960), 'every packet is 20 ms of decodable Opus');
|
||||
assert.ok(decoded.some(pcm => pcm!.some(v => Math.abs(v) > 0.01)), 'and carries the test tone, not silence');
|
||||
// Hearing: her own packets sent back as bob light him up, and no codec error shows
|
||||
const back = fromCarol.map((p, i) => encodeVoice({ target: 0, frame: 9000 + i * 2, opus: p.opus, last: i === fromCarol.length - 1 }, bob.protobufVoice));
|
||||
const pump = setInterval(() => { const pk = back.shift(); if (pk) bob.sendVoiceTunnel(pk); }, 20);
|
||||
await page2.locator('.sidebar .row.user', { hasText: 'bob' }).locator('.label.talking').waitFor({ timeout: 5000 });
|
||||
clearInterval(pump);
|
||||
assert.equal(await page2.getByText('Voice is not working').count(), 0);
|
||||
console.log(`ok: voice with the built-in Opus codec (${fromCarol[0].opus.length} bytes per packet)`);
|
||||
|
||||
await page.locator('.me').getByRole('button', { name: 'Share your screen' }).click();
|
||||
await page.getByRole('button', { name: 'Continue' }).click();
|
||||
const dialog = page.getByRole('dialog', { name: 'Share your screen' });
|
||||
await dialog.getByRole('button', { name: 'Choose a screen or window...' }).click();
|
||||
await page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 60000 });
|
||||
await dialog.getByRole('button', { name: 'Start sharing' }).click();
|
||||
await page.locator('.stage .status', { hasText: '0 watching' }).waitFor();
|
||||
// The viewer is limited to what a browser without direct UDP has (Vanadium's default): only
|
||||
// the proxy's relay, reached over TCP
|
||||
await page2.evaluate(() => localStorage.setItem('mumh5.iceDebug', 'relay-tcp'));
|
||||
await page2.locator('.stage .tile', { hasText: name }).click();
|
||||
await page2.getByRole('button', { name: 'Continue' }).click();
|
||||
await page2.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 30000 });
|
||||
await page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
|
||||
// Phone-sized window: the picture still has room and frames
|
||||
await page2.setViewportSize({ width: 390, height: 780 });
|
||||
await page2.waitForFunction(() => {
|
||||
const v = document.querySelector('.spot video') as HTMLVideoElement | null;
|
||||
return !!v && v.videoWidth > 0 && !v.paused && v.getBoundingClientRect().height > 100 && v.getBoundingClientRect().width > 300;
|
||||
}, null, { timeout: 15000 });
|
||||
if (process.env.SHOTS_DIR) await page2.screenshot({ path: `${process.env.SHOTS_DIR}/phone.png` });
|
||||
// The statistics confirm which way the stream came
|
||||
await page2.setViewportSize({ width: 1280, height: 800 });
|
||||
await page2.locator('.spot video').click({ button: 'right' });
|
||||
await page2.getByRole('menuitem', { name: 'Stats for nerds' }).click();
|
||||
const stats = page2.getByRole('status', { name: 'Stream statistics' });
|
||||
await stats.getByText(/^(direct|relayed)/).first().waitFor({ timeout: 8000 });
|
||||
const route = (await stats.innerText()).replace(/\n+/g, ' | ');
|
||||
assert.match(route, /relayed \(TURN\), reached over TCP/, route);
|
||||
assert.match(route, /0 direct, 0 through STUN, [1-9]\d* relayed/, route);
|
||||
await stats.getByText(/^\d+ kbit\/s$/).first().waitFor({ timeout: 8000 });
|
||||
console.log('ok: screen sharing to a browser without direct UDP, through the relay over TCP');
|
||||
|
||||
// A camera instead, still watched through the relay over TCP
|
||||
await page.locator('.me').getByRole('button', { name: 'Stop sharing your screen' }).click();
|
||||
await page2.getByRole('region', { name: 'Screen share' }).waitFor({ state: 'detached' });
|
||||
await page.locator('.me').getByRole('button', { name: 'Share your screen or camera' }).click();
|
||||
await dialog.getByRole('tab', { name: 'Camera' }).click();
|
||||
await page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 30000 });
|
||||
await dialog.getByRole('button', { name: 'Start sharing' }).click();
|
||||
await page2.locator('.sidebar').getByRole('button', { name: `Watch the camera of ${name}`, exact: true }).click();
|
||||
await page2.waitForFunction(() => (document.querySelector('.stage .tile video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 30000 });
|
||||
await page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
|
||||
console.log('ok: camera shared between two browsers, shown in the tile, through the relay');
|
||||
|
||||
console.log('WEB E2E PASSED');
|
||||
} finally {
|
||||
bob.disconnect();
|
||||
await app.close().catch(() => {});
|
||||
await second?.close().catch(() => {});
|
||||
proxy.kill();
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { instantiateOpus, WasmEncoder, WasmDecoder } from '../src/lib/audio/opus-wasm.ts';
|
||||
|
||||
const opus = await instantiateOpus(readFileSync(new URL('../node_modules/@evan/opus/wasm/opus.wasm', import.meta.url)));
|
||||
|
||||
// One second of a 440 Hz tone in the engine's 10 ms frames
|
||||
const tone = () => Array.from({ length: 100 }, (_, f) => Float32Array.from({ length: 480 }, (_, i) => 0.5 * Math.sin(2 * Math.PI * 440 * (f * 480 + i) / 48000)));
|
||||
|
||||
for (const frameMs of [10, 20, 40, 60]) {
|
||||
test(`a tone survives encoding and decoding with ${frameMs} ms packets`, () => {
|
||||
const packets: Uint8Array[] = [];
|
||||
const encoder = new WasmEncoder(opus, 40000, frameMs, p => packets.push(p));
|
||||
for (const frame of tone()) encoder.push(frame);
|
||||
assert.equal(packets.length, Math.floor(1000 / frameMs), 'one packet per frame duration');
|
||||
// Around the asked bitrate, not raw audio
|
||||
const bytes = packets.reduce((n, p) => n + p.length, 0);
|
||||
assert.ok(bytes > 2000 && bytes < 8000, `about 40 kbit/s: ${bytes} bytes for one second`);
|
||||
|
||||
const decoder = new WasmDecoder(opus);
|
||||
const out = packets.map(p => decoder.decode(p)!);
|
||||
assert.ok(out.every(pcm => pcm.length === 48 * frameMs), 'each packet decodes to its duration');
|
||||
// The tone is still there: count the zero crossings in the second half (the codec needs a moment to settle)
|
||||
const all = Float32Array.from(out.slice(out.length / 2).flatMap(pcm => [...pcm]));
|
||||
let crossings = 0;
|
||||
for (let i = 1; i < all.length; i++) if ((all[i - 1] < 0) !== (all[i] < 0)) crossings++;
|
||||
const hz = crossings / 2 / (all.length / 48000);
|
||||
assert.ok(Math.abs(hz - 440) < 15, `decoded tone at ${hz.toFixed(1)} Hz`);
|
||||
const peak = all.reduce((m, v) => Math.max(m, Math.abs(v)), 0);
|
||||
assert.ok(peak > 0.3 && peak < 0.8, `level kept: ${peak.toFixed(2)}`);
|
||||
encoder.close();
|
||||
decoder.close();
|
||||
});
|
||||
}
|
||||
|
||||
test('an unfinished frame is dropped at the end, and garbage does not decode', () => {
|
||||
const packets: Uint8Array[] = [];
|
||||
const encoder = new WasmEncoder(opus, 40000, 20, p => packets.push(p));
|
||||
encoder.push(tone()[0]);
|
||||
assert.equal(packets.length, 0);
|
||||
encoder.flush();
|
||||
encoder.push(tone()[0]);
|
||||
assert.equal(packets.length, 0, 'the half frame before the flush is gone');
|
||||
encoder.push(tone()[1]);
|
||||
assert.equal(packets.length, 1);
|
||||
const decoder = new WasmDecoder(opus);
|
||||
assert.equal(decoder.decode(new Uint8Array(5000)), null);
|
||||
assert.equal(decoder.decode(Uint8Array.of()), null);
|
||||
encoder.close();
|
||||
encoder.push(tone()[0]);
|
||||
decoder.close();
|
||||
assert.equal(decoder.decode(packets[0]), null);
|
||||
});
|
||||
@@ -0,0 +1,47 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { audioApps, planLinks, VIRTUAL_MIC } from '../electron/pipewire.ts';
|
||||
|
||||
// A cut-down pw-dump: the virtual microphone (10), this app (20, process 500), a browser whose
|
||||
// stream has no name of its own (30, named by its client), a mono game (40) and a music player (50)
|
||||
const node = (id: number, props: Record<string, unknown>) => ({ id, type: 'PipeWire:Interface:Node', info: { props } });
|
||||
const client = (id: number, name: string, pid: number) => ({ id, type: 'PipeWire:Interface:Client', info: { props: { 'application.name': name, 'application.process.id': pid } } });
|
||||
const port = (id: number, nodeId: number, direction: string, channel: string) => ({ id, type: 'PipeWire:Interface:Port', info: { direction, props: { 'node.id': nodeId, 'audio.channel': channel } } });
|
||||
const link = (id: number, outPort: number, inPort: number, inNode: number) =>
|
||||
({ id, type: 'PipeWire:Interface:Link', info: { 'output-port-id': outPort, 'input-port-id': inPort, 'input-node-id': inNode } as any });
|
||||
|
||||
const graph = [
|
||||
node(10, { 'node.name': VIRTUAL_MIC, 'media.class': 'Audio/Source/Virtual' }), port(11, 10, 'input', 'FL'), port(12, 10, 'input', 'FR'),
|
||||
port(13, 10, 'output', 'FL'), port(14, 10, 'output', 'FR'),
|
||||
client(2, 'mumh5', 500), node(20, { 'media.class': 'Stream/Output/Audio', 'node.name': 'mumh5', 'client.id': 2 }), port(21, 20, 'output', 'FL'), port(22, 20, 'output', 'FR'),
|
||||
client(3, 'Firefox', 600), node(30, { 'media.class': 'Stream/Output/Audio', 'node.name': 'Firefox', 'client.id': 3 }), port(31, 30, 'output', 'FL'), port(32, 30, 'output', 'FR'),
|
||||
node(40, { 'media.class': 'Stream/Output/Audio', 'application.name': 'Game', 'application.process.id': 700 }), port(41, 40, 'output', 'MONO'),
|
||||
node(50, { 'media.class': 'Stream/Output/Audio', 'application.name': 'Spotify' }), port(51, 50, 'output', 'FL'), port(52, 50, 'output', 'FR'), port(53, 50, 'output', 'LFE'),
|
||||
node(60, { 'media.class': 'Audio/Sink', 'node.name': 'speakers' })
|
||||
];
|
||||
|
||||
test('lists programs that play sound, without this app', () => {
|
||||
assert.deepEqual(audioApps(graph, [500]), ['Firefox', 'Game', 'Spotify']);
|
||||
});
|
||||
|
||||
test('whole system links every program but this app', () => {
|
||||
const { add, remove } = planLinks(graph, { kind: 'all' }, [500]);
|
||||
assert.deepEqual(add.sort(), [[31, 11], [32, 12], [41, 11], [41, 12], [51, 11], [52, 12]].sort());
|
||||
assert.deepEqual(remove, []);
|
||||
});
|
||||
|
||||
test('one program links only that program', () => {
|
||||
assert.deepEqual(planLinks(graph, { kind: 'app', name: 'Firefox' }, [500]).add, [[31, 11], [32, 12]]);
|
||||
});
|
||||
|
||||
test('existing links are kept, stale ones removed', () => {
|
||||
const linked = [...graph, link(90, 31, 11, 10), link(91, 51, 11, 10), link(92, 21, 60, 60)];
|
||||
const { add, remove } = planLinks(linked, { kind: 'app', name: 'Firefox' }, [500]);
|
||||
assert.deepEqual(add, [[32, 12]]);
|
||||
// Spotify's link goes; the link to the speakers is none of our business
|
||||
assert.deepEqual(remove, [91]);
|
||||
});
|
||||
|
||||
test('nothing without the virtual microphone', () => {
|
||||
assert.deepEqual(planLinks(graph.slice(5), { kind: 'all' }, []), { add: [], remove: [] });
|
||||
});
|
||||
+63
-2
@@ -2,8 +2,11 @@
|
||||
// real Mumble server and are skipped unless MUMBLE_TEST_HOST is set (see server.test.ts).
|
||||
import { test } from 'node:test';
|
||||
import net from 'node:net';
|
||||
import dgram from 'node:dgram';
|
||||
import assert from 'node:assert/strict';
|
||||
import { proxyLine } from '../electron/tls-transport.ts';
|
||||
import { stunResponse } from '../server/proxy.ts';
|
||||
import { configFromEnv } from '../server/proxy.ts';
|
||||
import { startProxy, defaults, parseServers, isPrivateAddress, type ProxyConfig } from '../server/proxy.ts';
|
||||
import { WebSocket as WsClient } from 'ws';
|
||||
import { WebSocketTransport } from '../src/core/ws-transport.ts';
|
||||
@@ -29,6 +32,13 @@ test('server list parsing', () => {
|
||||
assert.deepEqual(parseServers(''), []);
|
||||
});
|
||||
|
||||
test('empty environment variables fall back to the defaults', () => {
|
||||
const c = configFromEnv({ MUMH5_PORT: '', MUMH5_BIND: '', MUMH5_STUN_PORT: '', MUMH5_STUN_BIND: '', MUMH5_MAX_CONNECTIONS: '', MUMH5_SERVERS: 'a.example', MUMH5_ALLOW_ANY: '' });
|
||||
assert.deepEqual([c.port, c.bind, c.stunPort, c.stunBind, c.maxConnections, c.allowAny], [8080, '127.0.0.1', 3478, '::', 200, false]);
|
||||
assert.equal(configFromEnv({ MUMH5_STUN_PORT: '0' }).stunPort, null);
|
||||
assert.equal(configFromEnv({ MUMH5_PORT: '8089' }).port, 8089);
|
||||
});
|
||||
|
||||
test('private address detection', () => {
|
||||
for (const a of ['127.0.0.1', '10.1.2.3', '192.168.1.1', '172.20.0.1', '169.254.1.1', '100.64.0.1', '::1', 'fd00::1', 'fe80::1', '::ffff:10.0.0.1']) assert.ok(isPrivateAddress(a), a);
|
||||
for (const a of ['8.8.8.8', '172.32.0.1', '2001:4860:4860::8888', '::ffff:8.8.8.8']) assert.ok(!isPrivateAddress(a), a);
|
||||
@@ -40,7 +50,7 @@ test('refuses to start without allowed servers', async () => {
|
||||
|
||||
test('config lists the allowed servers', async () => {
|
||||
await withProxy({ servers: parseServers('voice.example.org=Friends') }, async base => {
|
||||
assert.deepEqual(await (await fetch(`${base}/api/config`)).json(), { servers: [{ host: 'voice.example.org', port: 64738, label: 'Friends' }], any: false });
|
||||
assert.deepEqual(await (await fetch(`${base}/api/config`)).json(), { servers: [{ host: 'voice.example.org', port: 64738, label: 'Friends' }], any: false, stun: null, turn: false });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -64,6 +74,56 @@ test('identity create, describe, export and import round trip', async () => {
|
||||
});
|
||||
});
|
||||
|
||||
// A binding request: type, length 0, magic cookie, 12 byte transaction id
|
||||
const stunRequest = () => Uint8Array.from([0, 1, 0, 0, 0x21, 0x12, 0xa4, 0x42, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]);
|
||||
|
||||
test('STUN answers carry the sender address, masked as the protocol asks', () => {
|
||||
const v4 = stunResponse(stunRequest(), '203.0.113.7', 54321)!;
|
||||
assert.deepEqual([...v4.subarray(0, 4)], [1, 1, 0, 12]);
|
||||
assert.deepEqual([...v4.subarray(4, 20)], [...stunRequest().subarray(4, 20)]);
|
||||
assert.deepEqual([...v4.subarray(20, 26)], [0, 0x20, 0, 8, 0, 1]);
|
||||
assert.equal(((v4[26] << 8) | v4[27]) ^ 0x2112, 54321);
|
||||
assert.deepEqual([...v4.subarray(28)].map((b, i) => b ^ [0x21, 0x12, 0xa4, 0x42][i]), [203, 0, 113, 7]);
|
||||
// An IPv4 sender seen through an IPv6 socket is still IPv4
|
||||
assert.deepEqual([...stunResponse(stunRequest(), '::ffff:203.0.113.7', 54321)!], [...v4]);
|
||||
const v6 = stunResponse(stunRequest(), '2001:db8::7', 1)!;
|
||||
assert.equal(v6[25], 2);
|
||||
const mask = stunRequest().subarray(4, 20);
|
||||
assert.deepEqual([...v6.subarray(28)].map((b, i) => b ^ mask[i]), [0x20, 0x01, 0x0d, 0xb8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7]);
|
||||
// Not a binding request
|
||||
assert.equal(stunResponse(new Uint8Array(20), '203.0.113.7', 1), null);
|
||||
assert.equal(stunResponse(stunRequest().subarray(0, 12), '203.0.113.7', 1), null);
|
||||
});
|
||||
|
||||
test('the proxy answers STUN over UDP and announces the port', async () => {
|
||||
const proxy = await startProxy({ ...defaults, port: 0, origins: ['*'], servers: parseServers('voice.example.org'), stunPort: 0, stunBind: '127.0.0.1' });
|
||||
try {
|
||||
assert.ok(proxy.stunPort);
|
||||
assert.equal((await (await fetch(`http://127.0.0.1:${proxy.port}/api/config`)).json()).stun, proxy.stunPort);
|
||||
const client = dgram.createSocket('udp4');
|
||||
const answer = await new Promise<Buffer>((resolve, reject) => {
|
||||
client.once('message', resolve);
|
||||
client.once('error', reject);
|
||||
client.send(stunRequest(), proxy.stunPort!, '127.0.0.1');
|
||||
setTimeout(() => reject(new Error('no STUN answer')), 3000);
|
||||
});
|
||||
assert.equal(((answer[26] << 8) | answer[27]) ^ 0x2112, client.address().port);
|
||||
// Relay credentials for people on this site: a username that expires and its password
|
||||
const base = `http://127.0.0.1:${proxy.port}`;
|
||||
assert.equal((await (await fetch(`${base}/api/config`)).json()).turn, true);
|
||||
// What clients need for screen sharing: the port, and relay credentials that expire.
|
||||
// Readable from any origin, since the desktop app asks from outside this site.
|
||||
const res = await fetch(`${base}/api/relay`, { headers: { Origin: 'app://mumh5' } });
|
||||
assert.equal(res.headers.get('access-control-allow-origin'), '*');
|
||||
const cred = await res.json();
|
||||
assert.ok(Number(cred.username) > Date.now() / 1000 && cred.credential.length > 20 && cred.port === proxy.stunPort);
|
||||
assert.deepEqual([...answer.subarray(28)].map((b, i) => b ^ [0x21, 0x12, 0xa4, 0x42][i]), [127, 0, 0, 1]);
|
||||
client.close();
|
||||
} finally {
|
||||
await proxy.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('requests from other origins are refused', async () => {
|
||||
await withProxy({ servers: parseServers('voice.example.org'), origins: [] }, async base => {
|
||||
const cross = await post(base, 'identity/create', { name: 'x' }, { Origin: 'https://evil.example' });
|
||||
@@ -181,7 +241,8 @@ test('two clients talk through the proxy', { skip: !target }, async () => {
|
||||
const a = await connect(`proxy-a-${Date.now() % 100000}`);
|
||||
const b = await connect(`proxy-b-${Date.now() % 100000}`);
|
||||
const got = new Promise<string>(resolve => b.on('text', m => resolve(m.html)));
|
||||
a.sendText({ channels: [0] }, 'hello through the proxy');
|
||||
// A direct message, so other test files listening in the root channel are not disturbed
|
||||
a.sendText({ users: [b.session!] }, 'hello through the proxy');
|
||||
assert.equal(await got, 'hello through the proxy');
|
||||
// No UDP in a browser: voice stays on the TCP tunnel
|
||||
assert.equal(a.udpOk, false);
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { SHARE, encodeShare, ShareAssembler } from '../src/core/share-signal.ts';
|
||||
|
||||
test('bare signals are one byte', async () => {
|
||||
const packets = await encodeShare(SHARE.on);
|
||||
assert.deepEqual(packets, [new Uint8Array([SHARE.on])]);
|
||||
assert.deepEqual(await new ShareAssembler().push(1, packets[0]), { type: SHARE.on });
|
||||
});
|
||||
|
||||
test('session descriptions survive chunking, in any order', async () => {
|
||||
// Random-looking text compresses badly, forcing several chunks
|
||||
const sdp = Array.from({ length: 400 }, (_, i) => `a=candidate:${(i * 2654435761 >>> 0).toString(36)} 1 udp ${i} 2001:db8::${i} typ host`).join('\r\n');
|
||||
const packets = await encodeShare(SHARE.offer, sdp);
|
||||
assert.ok(packets.length > 1, 'needs more than one packet');
|
||||
// Murmur drops plugin data over 1000 bytes
|
||||
assert.ok(packets.every(p => p.length <= 1000));
|
||||
const assembler = new ShareAssembler();
|
||||
let result = null;
|
||||
for (const p of [...packets].reverse()) result = await assembler.push(7, p);
|
||||
assert.deepEqual(result, { type: SHARE.offer, text: sdp });
|
||||
});
|
||||
|
||||
test('chunks from different senders do not mix', async () => {
|
||||
const a = await encodeShare(SHARE.answer, Array.from({ length: 400 }, (_, i) => (i * 40503 * 2654435761 >>> 0).toString(36)).join(' '));
|
||||
assert.ok(a.length > 1);
|
||||
const assembler = new ShareAssembler();
|
||||
assert.equal(await assembler.push(1, a[0]), null);
|
||||
for (const p of a.slice(1)) assert.equal(await assembler.push(2, p), null);
|
||||
let result = null;
|
||||
for (const p of a.slice(1)) result = await assembler.push(1, p);
|
||||
assert.equal(result?.type, SHARE.answer);
|
||||
});
|
||||
|
||||
test('garbage is ignored', async () => {
|
||||
const assembler = new ShareAssembler();
|
||||
assert.equal(await assembler.push(1, new Uint8Array([99])), null);
|
||||
assert.equal(await assembler.push(1, new Uint8Array([SHARE.offer, 1])), null);
|
||||
assert.equal(await assembler.push(1, new Uint8Array([SHARE.offer, 1, 5, 2, 1, 2, 3])), null);
|
||||
// A complete transfer that is not valid compressed data
|
||||
assert.equal(await assembler.push(1, new Uint8Array([SHARE.offer, 1, 0, 1, 255, 255, 255, 255])), null);
|
||||
});
|
||||
@@ -0,0 +1,218 @@
|
||||
// The relay (TURN) in the web proxy, driven by a small client written here: the same steps a
|
||||
// browser takes, over UDP and over TCP.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import dgram from 'node:dgram';
|
||||
import net from 'node:net';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { startTurn, turnCredential, build, parse, xorAddress, unxorAddress, type TurnOptions } from '../server/turn.ts';
|
||||
|
||||
const SECRET = 'test-secret';
|
||||
const options = (over: Partial<TurnOptions> = {}): TurnOptions => ({
|
||||
port: 0, bind: '127.0.0.1', relay: true, publicIp: '127.0.0.1', minPort: 41000, maxPort: 41999, maxAllocations: 10, maxPerAddress: 4, peerAllowed: () => true, ...over
|
||||
});
|
||||
|
||||
let counter = 0;
|
||||
const header = () => { const h = new Uint8Array(20); new DataView(h.buffer).setUint32(4, 0x2112a442); h[19] = ++counter; h[18] = counter >> 8; return h; };
|
||||
const text = (s: string) => new TextEncoder().encode(s);
|
||||
const attr = (m: NonNullable<ReturnType<typeof parse>>, type: number) => m.attrs.find(a => a.type === type)?.value;
|
||||
const code = (m: NonNullable<ReturnType<typeof parse>>) => { const e = attr(m, 0x0009); return e ? e[2] * 100 + e[3] : 0; };
|
||||
|
||||
// One client connection to the relay, UDP or TCP
|
||||
async function connect(port: number, transport: 'udp' | 'tcp') {
|
||||
const inbox: Uint8Array[] = [];
|
||||
const waiting: ((m: Uint8Array) => void)[] = [];
|
||||
const deliver = (m: Uint8Array) => { const w = waiting.shift(); if (w) w(m); else inbox.push(m); };
|
||||
const next = () => new Promise<Uint8Array>((resolve, reject) => {
|
||||
const m = inbox.shift();
|
||||
if (m) return resolve(m);
|
||||
const timer = setTimeout(() => reject(new Error('no answer from the relay')), 3000);
|
||||
waiting.push(v => { clearTimeout(timer); resolve(v); });
|
||||
});
|
||||
if (transport === 'udp') {
|
||||
const socket = dgram.createSocket('udp4');
|
||||
socket.on('message', deliver);
|
||||
await new Promise<void>(r => socket.bind(0, '127.0.0.1', r));
|
||||
return { next, send: (b: Uint8Array) => socket.send(b, port, '127.0.0.1'), close: () => socket.close() };
|
||||
}
|
||||
const socket = net.connect(port, '127.0.0.1');
|
||||
await new Promise<void>((r, j) => { socket.once('connect', () => r()); socket.once('error', j); });
|
||||
let pending: Buffer = Buffer.alloc(0);
|
||||
socket.on('data', (chunk: Buffer) => {
|
||||
pending = Buffer.concat([pending, chunk]);
|
||||
while (pending.length >= 4) {
|
||||
const length = ((pending[0] & 0xc0) === 0 ? 20 : 4) + pending.readUInt16BE(2);
|
||||
const framed = (length + 3) & ~3;
|
||||
if (pending.length < framed) break;
|
||||
deliver(Uint8Array.from(pending.subarray(0, length)));
|
||||
pending = pending.subarray(framed);
|
||||
}
|
||||
});
|
||||
return {
|
||||
next,
|
||||
send: (b: Uint8Array) => { const padded = (b.length + 3) & ~3; socket.write(Buffer.concat([b, Buffer.alloc(padded - b.length)])); },
|
||||
close: () => socket.destroy()
|
||||
};
|
||||
}
|
||||
|
||||
// Allocate: asked for credentials first, then granted
|
||||
async function allocate(c: Awaited<ReturnType<typeof connect>>, secret = SECRET) {
|
||||
const transport: [number, Uint8Array] = [0x0019, Uint8Array.from([17, 0, 0, 0])];
|
||||
c.send(build(0x0003, header(), [transport]));
|
||||
const challenge = parse(await c.next())!;
|
||||
assert.equal(challenge.type, 0x0113);
|
||||
assert.equal(code(challenge), 401);
|
||||
const realm = attr(challenge, 0x0014)!, nonce = attr(challenge, 0x0015)!;
|
||||
const username = String(Math.floor(Date.now() / 1000) + 600);
|
||||
const key = createHash('md5').update(`${username}:mumh5:${turnCredential(secret, username)}`).digest();
|
||||
const auth: [number, Uint8Array][] = [[0x0006, text(username)], [0x0014, realm], [0x0015, nonce]];
|
||||
const h = header();
|
||||
c.send(build(0x0003, h, [transport, ...auth], key));
|
||||
const answer = parse(await c.next())!;
|
||||
return { answer, key, auth, relay: answer.type === 0x0103 ? unxorAddress(attr(answer, 0x0016)!, h) : null };
|
||||
}
|
||||
|
||||
for (const transport of ['udp', 'tcp'] as const) {
|
||||
test(`relays between a client and a peer over ${transport}`, async () => {
|
||||
const turn = await startTurn(options(), SECRET);
|
||||
const client = await connect(turn.port, transport);
|
||||
const peer = dgram.createSocket('udp4');
|
||||
await new Promise<void>(r => peer.bind(0, '127.0.0.1', r));
|
||||
const peerPort = peer.address().port;
|
||||
const atPeer: { data: string; port: number }[] = [];
|
||||
let peerGot: (() => void) | null = null;
|
||||
peer.on('message', (m, from) => { atPeer.push({ data: m.toString(), port: from.port }); peerGot?.(); });
|
||||
const peerNext = () => atPeer.length ? Promise.resolve() : new Promise<void>((r, j) => { peerGot = r; setTimeout(() => j(new Error('peer got nothing')), 3000); });
|
||||
try {
|
||||
const { answer, key, auth, relay } = await allocate(client);
|
||||
assert.equal(answer.type, 0x0103);
|
||||
assert.equal(relay!.ip, '127.0.0.1');
|
||||
assert.ok(relay!.port >= 41000 && relay!.port <= 41999, 'relay port within the configured range');
|
||||
// The answer is signed, which browsers insist on
|
||||
assert.ok(attr(answer, 0x0008), 'message integrity on the answer');
|
||||
|
||||
// Without permission, nothing gets through in either direction
|
||||
let h = header();
|
||||
const peerAddr = (hd: Uint8Array): [number, Uint8Array] => [0x0012, xorAddress('127.0.0.1', peerPort, hd)!];
|
||||
client.send(build(0x0016, h, [peerAddr(h), [0x0013, text('too early')]]));
|
||||
peer.send('unwanted', relay!.port, '127.0.0.1');
|
||||
// Let both arrive (and be dropped) before permission is given
|
||||
await new Promise(r => setTimeout(r, 150));
|
||||
assert.equal(atPeer.length, 0, 'nothing reaches the peer without permission');
|
||||
|
||||
h = header();
|
||||
client.send(build(0x0008, h, [peerAddr(h), ...auth], key));
|
||||
assert.equal(parse(await client.next())!.type, 0x0108);
|
||||
|
||||
// Send indication out, data indication back
|
||||
h = header();
|
||||
client.send(build(0x0016, h, [peerAddr(h), [0x0013, text('hello peer')]]));
|
||||
await peerNext();
|
||||
assert.deepEqual(atPeer.map(p => p.data), ['hello peer']);
|
||||
assert.equal(atPeer[0].port, relay!.port, 'the peer sees the relay as the sender');
|
||||
peer.send('hello client', relay!.port, '127.0.0.1');
|
||||
const indication = parse(await client.next())!;
|
||||
assert.equal(indication.type, 0x0017);
|
||||
assert.equal(new TextDecoder().decode(attr(indication, 0x0013)!), 'hello client');
|
||||
assert.deepEqual(unxorAddress(attr(indication, 0x0012)!, indication.header), { ip: '127.0.0.1', port: peerPort });
|
||||
|
||||
// Channel: the compact framing browsers switch to for media
|
||||
h = header();
|
||||
client.send(build(0x0009, h, [[0x000c, Uint8Array.from([0x40, 0x01, 0, 0])], peerAddr(h), ...auth], key));
|
||||
assert.equal(parse(await client.next())!.type, 0x0109);
|
||||
atPeer.length = 0;
|
||||
client.send(Uint8Array.from([0x40, 0x01, 0, 5, ...text('media')]));
|
||||
await peerNext();
|
||||
assert.equal(atPeer[0].data, 'media');
|
||||
peer.send('frames', relay!.port, '127.0.0.1');
|
||||
const data = await client.next();
|
||||
assert.deepEqual([...data.subarray(0, 4)], [0x40, 0x01, 0, 6]);
|
||||
assert.equal(new TextDecoder().decode(data.subarray(4)), 'frames');
|
||||
|
||||
// Refresh with lifetime 0 ends the allocation
|
||||
h = header();
|
||||
client.send(build(0x0004, h, [[0x000d, Uint8Array.from([0, 0, 0, 0])], ...auth], key));
|
||||
assert.equal(parse(await client.next())!.type, 0x0104);
|
||||
h = header();
|
||||
client.send(build(0x0008, h, [peerAddr(h), ...auth], key));
|
||||
assert.equal(code(parse(await client.next())!), 437);
|
||||
} finally {
|
||||
client.close();
|
||||
peer.close();
|
||||
turn.close();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
test('two clients of the relay reach each other at its announced address, even when that is not on this machine', async () => {
|
||||
// As behind a 1:1 NAT: the address handed out is the public one, which the host itself cannot loop through
|
||||
const turn = await startTurn(options({ publicIp: '203.0.113.9', peerAllowed: ip => !ip.startsWith('127.') }), SECRET);
|
||||
const a = await connect(turn.port, 'tcp'), b = await connect(turn.port, 'udp');
|
||||
try {
|
||||
const ra = await allocate(a), rb = await allocate(b);
|
||||
assert.equal(ra.relay!.ip, '203.0.113.9');
|
||||
for (const [me, creds, other] of [[a, ra, rb], [b, rb, ra]] as const) {
|
||||
const h = header();
|
||||
me.send(build(0x0008, h, [[0x0012, xorAddress('203.0.113.9', other.relay!.port, h)!], ...creds.auth], creds.key));
|
||||
assert.equal(parse(await me.next())!.type, 0x0108);
|
||||
}
|
||||
const h = header();
|
||||
a.send(build(0x0016, h, [[0x0012, xorAddress('203.0.113.9', rb.relay!.port, h)!], [0x0013, text('between clients')]]));
|
||||
const got = parse(await b.next())!;
|
||||
assert.equal(got.type, 0x0017);
|
||||
assert.equal(new TextDecoder().decode(attr(got, 0x0013)!), 'between clients');
|
||||
// It appears to come from the other client's relayed address, not from loopback
|
||||
assert.deepEqual(unxorAddress(attr(got, 0x0012)!, got.header), { ip: '203.0.113.9', port: ra.relay!.port });
|
||||
} finally {
|
||||
a.close();
|
||||
b.close();
|
||||
turn.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('wrong credentials and forbidden peers are refused', async () => {
|
||||
const turn = await startTurn(options({ peerAllowed: ip => ip !== '10.1.2.3' }), SECRET);
|
||||
const client = await connect(turn.port, 'udp');
|
||||
try {
|
||||
const bad = await allocate(client, 'another-secret');
|
||||
assert.equal(code(bad.answer), 401);
|
||||
const { answer, key, auth } = await allocate(client);
|
||||
assert.equal(answer.type, 0x0103);
|
||||
const h = header();
|
||||
client.send(build(0x0008, h, [[0x0012, xorAddress('10.1.2.3', 9, h)!], ...auth], key));
|
||||
assert.equal(code(parse(await client.next())!), 403);
|
||||
// A second allocation for the same client
|
||||
const again = await allocate(client);
|
||||
assert.equal(code(again.answer), 437);
|
||||
} finally {
|
||||
client.close();
|
||||
turn.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('quota per address', async () => {
|
||||
const turn = await startTurn(options({ maxPerAddress: 1 }), SECRET);
|
||||
const a = await connect(turn.port, 'udp'), b = await connect(turn.port, 'udp');
|
||||
try {
|
||||
assert.equal((await allocate(a)).answer.type, 0x0103);
|
||||
assert.equal(code((await allocate(b)).answer), 486);
|
||||
} finally {
|
||||
a.close();
|
||||
b.close();
|
||||
turn.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('STUN only: binding is answered, allocation is ignored', async () => {
|
||||
const turn = await startTurn(options({ relay: false }), SECRET);
|
||||
const client = await connect(turn.port, 'udp');
|
||||
try {
|
||||
client.send(build(0x0001, header(), []));
|
||||
assert.equal(parse(await client.next())!.type, 0x0101);
|
||||
client.send(build(0x0003, header(), [[0x0019, Uint8Array.from([17, 0, 0, 0])]]));
|
||||
await assert.rejects(client.next(), /no answer/);
|
||||
} finally {
|
||||
client.close();
|
||||
turn.close();
|
||||
}
|
||||
});
|
||||
Reference in New Issue
Block a user