mirror of
https://github.com/nestriness/nestri.git
synced 2026-11-03 14:12:39 +02:00
Wanjohi
f889516ef4
The box half of phase 1 of window streams: two games in one box, each its own stream. Wire formats are nesprotocol's `window` module (#421). ## What each app does now **nescope** - Gives every toplevel an id on first map: 0 for the first window, never reused, none past 65535. - Tells the hub, on the existing input socket, when a window opens, changes title or size, and closes, in nesprotocol's window encoding. It also sends a box-internal `MSG_WINDOW_PID` saying which process owns the window: the Wayland client, or `_NET_WM_PID` for Xwayland. The launch id comes from that process's `NESTRI_LAUNCH_ID`. - Handles `INPUT_WINDOW_FOCUS` by raising and focusing that window, and `INPUT_WINDOW_OUTPUT_SIZE` per window. The old `INPUT_OUTPUT_SIZE` means window 0. There is still one output: a window's size applies while that window has focus and is remembered otherwise. On every input connection, input resets to window 0. - Before cursor updates, sends `MSG_CURSOR_WINDOW` when the window they describe changes. **nescapture** - Tags each video IPC frame with its pid in a trailer (`IPC_FLAG_PID` in the reserved byte). An older reader never sees it. - Listens on `/tmp/nescapture-cmd-<pid>.sock`. It still binds the shared `/tmp/nescapture-cmd.sock` and relays from it, so whatever sends there still reaches the newest encoder, as before. The shared socket was the only resource two capturing processes fought over: the second one took it, and commands went only to that one. The video and stats sockets are send-only datagram sockets and are safe to share. Capture slots and devices are per process. **neshub** - Joins frames to windows by pid. An untagged frame is window 0. A pid with no window falls back to the newest window of its launch, then to window 0 while no other window is open. A pid whose window closed belongs to nobody. - A connection that never subscribes gets the box's primary picture in the old encoding. The primary picture is the frames of the first encoder to draw, until it has been silent for a second. A subscribed connection gets only its window's frames, windowed for any window but 0, with per-window seq. PyroWave stays window 0 only. - `MSG_WINDOW_SUBSCRIBE` switches the connection first, then resets the writer's seq and resync gate in line with the frames, then requests a keyframe. The first subscribe opens `STREAM_WINDOWS` on the video connection, starting with `OPENED` (and the icon, if known) for every open window. Later lifecycle events go to every subscribed connection. - Input to nescope carries `INPUT_WINDOW_FOCUS` only when the window changes, and the hub does no merging across tags. A gamepad message moves focus only when its window differs from the current one and no other client sent keyboard or pointer input in the last second. Focus tags move keyboard and pointer focus only: nescope neither raises the window nor resizes the output for one. The output changes size only on an explicit `INPUT_WINDOW_OUTPUT_SIZE` from the window that owns it, last writer wins with 500 ms hysteresis between windows. A client's `INPUT_OUTPUT_SIZE` is rewritten to `INPUT_WINDOW_OUTPUT_SIZE`, except for window 0. - A cursor update goes to the clients subscribed to the window it describes. Clients that never subscribed get every update, as before. - When nescope reconnects (a new compositor), the hub sends `CLOSED` for every window and every subscription goes dead until the client subscribes again. - `DatagramSender` sizes fragments with `max_fragment_payload` and writes headers with `write_video_datagram_header`. - Commands go to every known encoder on its own socket, or to the shared socket while none is known (as today). An encoder that appears later is sent the commands so far (the latest of each message type) and then a keyframe request. The first encoder is not sent them again. - A keyframe on a client's loss or subscribe goes only to that client's window's encoder (the primary's, for a client that never subscribed). - Known limit: bitrate is box-wide. One controller sets one target for every window's encoder, so one client's congestion sets every window's rate. Per-window rate is follow-up work. - An X11 unmap is `CLOSED` for the hub, and a re-map is `OPENED` with the same id. nescope no longer exits when its first game exits; nesinit owns box power-off. The hub never reads `/proc/<pid>/environ` (it runs as another uid): nescapture sends its launch id. ## Shipping nescope and neshub ship together in the guest image and must not ship apart. ## Compatibility One game with an old client sends the same bytes as before. The client never subscribes, so it gets the primary picture, which is that game, as `DGRAM_VIDEO`/`MSG_DATA` with an unchanged seq. It gets no windows stream. Cursor updates reach it as before. Focus tags appear only if the game's window is not 0, and window 0 resizes use the old form. Mixed versions also work: - An old nescapture is window 0. - An old nescope never sends window messages. Every frame is then window 0, or the primary picture. - An old hub sends no tags, which nescope reads as window 0, and it still reaches nescapture through the shared socket. ## Tests `cargo test` on nesprotocol (122), nescope (17+5), nescapture (161), neshub (74), all passing. New tests: - The window table: joining pids to windows, the launch fallback, retired pids, the primary picture, the snapshot. - Focus tagging and resize rewriting. - The command fan-out. - nescope's id registry (first window 0, ids never reused, refused past 65535) and per-window focus and sizes. - The IPC pid trailer. - Two QUIC end-to-end tests: - Two tagged encoders: an unsubscribed client sees only the first game in the old encoding. After subscribing to window 1 it sees only the second game, windowed, with seq starting again, and a windows stream that starts with both `OPENED` events. - An untagged encoder gives the old stream. Not run on a box: no neslet, nestri-host or box was touched.