nescope used `EnvFilter::from_default_env()`, whose default is ERROR, so
its info/warn lines never reached the host journal. nesinit only
forwards RUST_LOG if it has it itself.
- nescope: fall back to `info` when RUST_LOG is unset or unparsable
(same as neshub, nesinit, nesgamepad); debug stays opt-in. 4 unit tests
on the filter helper.
- neswire: same ERROR default, same fix.
- Not changed: nescapture (library inside the game process, env_logger).
Tested on nestripc-1: `cargo test -p nescope log_filter` 4 passed,
`cargo check -p neswire` ok.
Note: no 'configure_request clamped' warn exists on this base; once it
lands at warn level it is covered by the info default.
EnvFilter::from_default_env defaults to ERROR, so nescope's info and warn
lines never reached the host journal. Fall back to info like neshub,
nesinit and nesgamepad; align neswire, which had the same default.
The input source now removes itself on EOF or reset and raises a flag; the
main loop exits 2 s later unless a signal already ended it, so a hub crash
mid-run still ends the compositor (and the box via CompositorDied) instead of
holding a GPU with no input. Also corrects the comment that promised a
reconnect that does not exist.
The input source returned UnixStream EOF as an error, which calloop passes
out of run() into the expect at the end of main. At box shutdown init
signals every process at once, so neshub can go before nescope reads its
SIGTERM: nescope panicked and skipped its cleanup. Deliver what was read,
remove the source and warn instead.
The box half of phase 1 of window streams: two games in one box, each
its own stream. Wire formats are nesprotocol's `window` module (#421).
## What each app does now
**nescope**
- Gives every toplevel an id on first map: 0 for the first window, never
reused, none past 65535.
- Tells the hub, on the existing input socket, when a window opens,
changes title or size, and closes, in nesprotocol's window encoding. It
also sends a box-internal `MSG_WINDOW_PID` saying which process owns the
window: the Wayland client, or `_NET_WM_PID` for Xwayland. The launch id
comes from that process's `NESTRI_LAUNCH_ID`.
- Handles `INPUT_WINDOW_FOCUS` by raising and focusing that window, and
`INPUT_WINDOW_OUTPUT_SIZE` per window. The old `INPUT_OUTPUT_SIZE` means
window 0. There is still one output: a window's size applies while that
window has focus and is remembered otherwise. On every input connection,
input resets to window 0.
- Before cursor updates, sends `MSG_CURSOR_WINDOW` when the window they
describe changes.
**nescapture**
- Tags each video IPC frame with its pid in a trailer (`IPC_FLAG_PID` in
the reserved byte). An older reader never sees it.
- Listens on `/tmp/nescapture-cmd-<pid>.sock`. It still binds the shared
`/tmp/nescapture-cmd.sock` and relays from it, so whatever sends there
still reaches the newest encoder, as before. The shared socket was the
only resource two capturing processes fought over: the second one took
it, and commands went only to that one. The video and stats sockets are
send-only datagram sockets and are safe to share. Capture slots and
devices are per process.
**neshub**
- Joins frames to windows by pid. An untagged frame is window 0. A pid
with no window falls back to the newest window of its launch, then to
window 0 while no other window is open. A pid whose window closed
belongs to nobody.
- A connection that never subscribes gets the box's primary picture in
the old encoding. The primary picture is the frames of the first encoder
to draw, until it has been silent for a second. A subscribed connection
gets only its window's frames, windowed for any window but 0, with
per-window seq. PyroWave stays window 0 only.
- `MSG_WINDOW_SUBSCRIBE` switches the connection first, then resets the
writer's seq and resync gate in line with the frames, then requests a
keyframe. The first subscribe opens `STREAM_WINDOWS` on the video
connection, starting with `OPENED` (and the icon, if known) for every
open window. Later lifecycle events go to every subscribed connection.
- Input to nescope carries `INPUT_WINDOW_FOCUS` only when the window
changes, and the hub does no merging across tags. A gamepad message
moves focus only when its window differs from the current one and no
other client sent keyboard or pointer input in the last second. Focus
tags move keyboard and pointer focus only: nescope neither raises the
window nor resizes the output for one. The output changes size only on
an explicit `INPUT_WINDOW_OUTPUT_SIZE` from the window that owns it,
last writer wins with 500 ms hysteresis between windows. A client's
`INPUT_OUTPUT_SIZE` is rewritten to `INPUT_WINDOW_OUTPUT_SIZE`, except
for window 0.
- A cursor update goes to the clients subscribed to the window it
describes. Clients that never subscribed get every update, as before.
- When nescope reconnects (a new compositor), the hub sends `CLOSED` for
every window and every subscription goes dead until the client
subscribes again.
- `DatagramSender` sizes fragments with `max_fragment_payload` and
writes headers with `write_video_datagram_header`.
- Commands go to every known encoder on its own socket, or to the shared
socket while none is known (as today). An encoder that appears later is
sent the commands so far (the latest of each message type) and then a
keyframe request. The first encoder is not sent them again.
- A keyframe on a client's loss or subscribe goes only to that client's
window's encoder (the primary's, for a client that never subscribed).
- Known limit: bitrate is box-wide. One controller sets one target for
every window's encoder, so one client's congestion sets every window's
rate. Per-window rate is follow-up work.
- An X11 unmap is `CLOSED` for the hub, and a re-map is `OPENED` with
the same id. nescope no longer exits when its first game exits; nesinit
owns box power-off. The hub never reads `/proc/<pid>/environ` (it runs
as another uid): nescapture sends its launch id.
## Shipping
nescope and neshub ship together in the guest image and must not ship
apart.
## Compatibility
One game with an old client sends the same bytes as before. The client
never subscribes, so it gets the primary picture, which is that game, as
`DGRAM_VIDEO`/`MSG_DATA` with an unchanged seq. It gets no windows
stream. Cursor updates reach it as before. Focus tags appear only if the
game's window is not 0, and window 0 resizes use the old form. Mixed
versions also work:
- An old nescapture is window 0.
- An old nescope never sends window messages. Every frame is then window
0, or the primary picture.
- An old hub sends no tags, which nescope reads as window 0, and it
still reaches nescapture through the shared socket.
## Tests
`cargo test` on nesprotocol (122), nescope (17+5), nescapture (161),
neshub (74), all passing. New tests:
- The window table: joining pids to windows, the launch fallback,
retired pids, the primary picture, the snapshot.
- Focus tagging and resize rewriting.
- The command fan-out.
- nescope's id registry (first window 0, ids never reused, refused past
65535) and per-window focus and sizes.
- The IPC pid trailer.
- Two QUIC end-to-end tests:
- Two tagged encoders: an unsubscribed client sees only the first game
in the old encoding. After subscribing to window 1 it sees only the
second game, windowed, with seq starting again, and a windows stream
that starts with both `OPENED` events.
- An untagged encoder gives the old stream.
Not run on a box: no neslet, nestri-host or box was touched.
Protocol for several windows in one box, each its own stream,
ref(d-0085). Design in `crates/nesprotocol/src/window.rs` module docs.
- Video: window 0 is byte-identical to today (DGRAM_VIDEO / MSG_DATA).
Other windows use DGRAM_VIDEO_WINDOW (4, 17-byte header with u16 window
id after sent_us) and MSG_WINDOW_DATA (0x01, u16 window id after seq).
- Lifecycle (hub -> client, new uni STREAM_WINDOWS = 8, length-prefixed
frames): OPENED 0x20, TITLE 0x21, ICON 0x22 (RGBA), SIZE 0x23, CLOSED
0x24.
- MSG_WINDOW_SUBSCRIBE 0x17 on the control stream; never sent means
window 0. Old hubs log it as unknown and continue.
- Box-internal input: INPUT_WINDOW_OUTPUT_SIZE (5) and
INPUT_WINDOW_FOCUS (6) tag resize and input with a window; client
INPUT_OUTPUT_SIZE unchanged.
- Cursor unchanged: per subscription.
nescope gains a no-op match arm for the two new DecodedInput variants.
Tests: cargo test -p nesprotocol --all-features, 140 passed. cargo check
--workspace (excluding nespyro/nescapture, slangc missing on the build
box) clean.
A new input event, INPUT_OUTPUT_SIZE [4][w u16][h u16], rides the input
path to nescope, which resizes its output and every window on it. The
size is clamped to the launch size, so the box's encode budget holds,
and kept even for 4:2:0. apply_resize existed but nothing called it.
Old hubs stop a batch at an unknown type, so clients send it alone.
A Vulkan client presenting FIFO waits in the driver for its frame
callback. Chromium's GPU process draws for a window another process
owns, which Wine's Wayland driver gives a surface with no role, so no
window here matched it and the game presented one frame and waited
forever: Sheepy's black screen. Nothing here paces against a display,
so every commit now gets its frame callbacks and feedback.
Since frame callbacks moved onto the commit path, the commit handler
found a window by its xdg toplevel. An Xwayland window has none, so no
Proton game got a frame callback or presentation feedback. Xwayland then
completes a flip only on its one-second fallback timer, and a game
presenting FIFO stalls in the driver after its first frame.
Get this thing going..
<!-- greptile_comment -->
<!-- greptile_summary -->
<h2><a
href="https://app.greptile.com/api/retrigger?id=63134761"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/RetriggerDark.svg?v=1"><source
media="(prefers-color-scheme: light)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1"><img
alt="Retrigger"
src="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1"
align="right"></picture></a>Confidence Score: 5/5</h2>
The PR appears safe to merge; all previous findings are resolved and the
latest readiness change introduces no established actionable regression.
<h3>Summary</h3>
- Establishes required guest filesystems, runtime directories, device
permissions, and service processes.
- Reports initialization and service deaths over the lifecycle channel.
- Supports launch, restart, and shutdown commands for a resident guest.
- Separates service and workload identities and configures per-launch
runtime environments.
- Removes the currently inactive nescope screenshot option and makes
capture-chain verification fail explicitly when compositor readback is
unavailable.
- Reworks the guest image around `nesinit` as PID 1 without a
distribution service manager.
<h3>Diagram</h3>
```mermaid
sequenceDiagram
participant Host
participant Init as nesinit
participant FS as Guest filesystems
participant Services as Service stack
participant Workload
Init->>Host: Ready(protocol version)
Host->>Init: Boot(mount descriptors)
Init->>FS: Establish and mount shares
Init->>Services: Spawn services in order
Services-->>Init: Required sockets ready
Init->>Host: Initialized(service names)
Host->>Init: Launch(id, exec, on_exit)
Init->>Workload: Spawn with isolated UID/runtime
Init->>Host: Started(id)
Workload-->>Init: Exit status
Init->>Host: WorkloadExited(id, status)
Host->>Init: Launch / Restart / Shutdown
```
<sub>Reviews (4) · Last reviewed commit: ["fix(nesinit): readiness is a
socket
that..."](https://github.com/nestrilabs/nestri/commit/731d34df9df30463f67963363424cb9487e89196)</sub>
<!-- /greptile_comment -->
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Three related fixes. Together they take HDR from unreachable to working
end to end on the XWayland path.
## XWayland was never started
Three lines had been commented out since the initial import: the call
that spawns XWayland, the guard that waits for it, and the `DISPLAY`
handed to the child. Every game therefore launched as a native Wayland
client. Nothing reported it -- the compositor still logged the X display
it was telling clients to point at, which is why it read as working.
That is also why HDR never fired. The colour space is signalled over a
protocol whose Vulkan layer lives in the game process and finds the
compositor through the X11 root window, so the one path able to carry it
was the one path no game was on. `ENABLE_GAMESCOPE_WSI` and `DXVK_HDR`
were already being set, which switched that layer on and then handed it
a
display it could not use.
Restoring the guard also fixes the ordering it was written for: the
launch
now happens after XWayland reports ready rather than ~40ms before it.
## Mesa was dropping every format we advertised alpha-only
Mesa tracks two flags per VkFormat -- one contributed by a format alpha
FourCC, one by its opaque FourCC -- and skips any format carrying only
one:
```c
if (!(disp_fmt->flags & WSI_WL_FMT_ALPHA) ||
!(disp_fmt->flags & WSI_WL_FMT_OPAQUE))
continue;
```
We advertised `ARGB8888` and `XRGB8888`, so `B8G8R8A8` survived and made
the list look like it was working. Everything else was alpha-only and
was
dropped in silence -- `ABGR8888` had been advertised all along while
`R8G8B8A8` never once appeared on a surface. Adding the opaque spellings
takes the surface from 6 formats to 21 and restores the three that carry
HDR.
The comment above that list claimed it was for XWayland DRI3 and that a
game swapchain format was independent of it. It was the opposite: the
list decides what a game can select, and deleting an entry removes that
format from every client.
## Verified against swapchains, not format lists
A client asking for `A2B10G10R10` + `HDR10_ST2084` now gets a swapchain
and the compositor is told colorspace `1000104008`; one asking for
`R16G16B16A16_SFLOAT` + scRGB linear gets `1000104002`. Previously both
were refused at creation -- the WSI layer re-checks the requested format
against the driver own surface list, so the colour space and the pixel
format arrive from two different places and only one was being supplied.
`apps/nescope/scripts/verify-hdr-formats.sh` asks what a client is
offered
from inside a child process, keeping the XCB and Wayland surfaces apart
since a game presents through the XCB one. The default mode guards both
halves of what the compositor controls; `--expect-layer` states the full
target and passes once a WSI layer is present. No new dependencies
(`vulkaninfo` + `python3`).
## Still open
HDR is XWayland-only, documented as a FIXME in `hdr.rs`. A WSI layer
binds
the swapchain factory on its own Wayland connection while a native
client
surface lives on the client one, and object IDs do not cross
connections.
The FIXME records the fix both reference implementations point at, and
the
trap to avoid when we take it: gating format injection on "the
compositor
supports HDR" rather than on being able to signal the surface hands a
client PQ pixels that arrive tagged as SDR, with nothing reporting an
error.
nescope ships no WSI layer of its own; the above was verified with the
stock gamescope one, which drives our protocol unmodified.
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
This PR makes HDR-capable native Wayland presentation reachable, adds
the alpha/opaque DMA-BUF FourCC pairs Mesa requires, makes XWayland an
explicit compatibility mode, and adds an HDR surface-format diagnostic.
- Starts XWayland only with `--xwayland`, waits for readiness before
launching the child, and stops cleanly if startup fails or times out.
- Routes Proton through Wayland unconditionally while retaining
`DXVK_HDR` as an HDR-specific setting.
- Advertises paired alpha and opaque FourCC variants with portable
modifiers.
- Separates XCB and Wayland probe results, selects one hardware adapter,
and distinguishes probe failures from format regressions.
- Documents the limitations of the legacy gamescope WSI path and the
external-layer dependency.
<h3>Confidence Score: 5/5</h3>
The PR appears safe to merge; no outstanding correctness, security, or
repository-rule issue remains.
All previous findings are resolved in the current code, including the
XWayland failure lifecycle, removal of unsupported vendor modifiers,
corrected HDR documentation, reliable diagnostic failure handling,
per-GPU format selection, and unconditional Proton Wayland routing. The
changes since the previous review preserve diagnostic output handling
without introducing a new failure.
<h3>Important Files Changed</h3>
| Filename | Overview |
|----------|----------|
| apps/nescope/src/main.rs | Adds opt-in XWayland lifecycle handling,
readiness timeout, conditional DISPLAY propagation, and unconditional
Proton Wayland routing; the previous launch-environment finding is
fixed. |
| apps/nescope/src/state.rs | Stops the event loop on reported XWayland
startup failure and advertises portable paired alpha/opaque DMA-BUF
formats without vendor-specific modifiers. |
| apps/nescope/src/hdr.rs | Documents the working native Wayland HDR
path and accurately distinguishes it from the external, deliberately
disabled gamescope WSI route. |
| apps/nescope/scripts/verify-hdr-formats.sh | Adds a diagnostic that
keeps GPU and surface paths separate and now preserves the intended exit
behavior when filtered Vulkan diagnostics contain no matching lines. |
<h3>Flowchart</h3>
```mermaid
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Launch[nescope child launch] --> Mode{--xwayland?}
Mode -->|No| Wayland[Native Wayland surface]
Mode -->|Yes| Wait[Start and await XWayland]
Wait -->|Ready| XCB[XCB / XWayland surface]
Wait -->|Error or 10s timeout| Stop[Log failure and stop]
Wayland --> Formats[Paired alpha and opaque FourCCs]
Formats --> HDR[HDR10 and scRGB formats available]
XCB --> SDR[X11 compatibility path without native HDR]
Proton[Proton child] -->|PROTON_ENABLE_WAYLAND=1| Wayland
```
<sub>Reviews (9): Last reviewed commit: ["nescope/scripts: guard the
diagnostic
pi..."](https://github.com/nestrilabs/nestri/commit/f8bdd68f87b3e29e4195f325b3c98800076a55d5)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=60368324)</sub>
**Context used:**
- Knowledge Base — [Compositor, display, and input
control](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/compositor-input.md)
- Knowledge Base — [Streaming host
runtime](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/streaming-runtime.md)
<!-- /greptile_comment -->
All three came across with their standalone repos and none had been
touched since the transport changed.
nescapture claimed to packetize with Reed-Solomon FEC and stream RTP/UDP
to a Moonlight client. It sends encoded frames to neshub over a Unix
socket. packetizer.rs, control.rs and shard_batch.rs do not exist, and
neither does ARCHITECTURE.md. Nine documented environment variables are
not read by anything -- NESCAPTURE_RTP_HOST was listed as *required* --
and five that are read were undocumented, including the one that says
where frames go. Someone following that quick start would have set a
required variable that does nothing and got no output.
Documented the three sockets, since nescapture binds one and connects to
two and that was written down nowhere.
neswire documented --rtp-addr and --channels against a gstreamer pipeline.
It has --ipc-path, --channels, --packet-duration-ms and
--bitrate-per-channel, and hub-stub exists precisely so it can be tested
without a hub. Kept the reason hub-stub decodes rather than counts bytes:
Opus codes silence at ~3 kbps, so a dead sink looks alive on a meter.
nescope was mostly right. It called the capture layer "vkcapture", listed
seven of fifteen modules and five of nine flags, and its TODO list was
three items that neshub and nescapture now do. Added compositor mode,
which is the shape a real session uses and was undocumented.
All three licence lines were "TBD" or "See project repository".
Six comments across nescope, neswire and nescapture still named
nestri-guest-hub, plus one still naming nestri-protocol. Deferred from the
import commits so the rename would read as one change rather than six
unexplained edits inside otherwise-verbatim trees.
Comments only. Realigned an ASCII box in encode.rs that the shorter name
knocked crooked.
A headless Wayland compositor for a single fullscreen client, and the second
component into this repo. Imported as a tree from `nestrilabs/nescope` for the
same reason as the last one: the upstream repo is private, its history has never
been reviewed for publication, and a squash is what keeps that history from
becoming permanent here.
Wired to the workspace — versions from the root, `nesprotocol` by path instead
of a sibling directory. 8 tests pass.
It knows a lot about Steam, and all of it stays. `steam_app_*` window classes,
a launcher that exits before the game it started, a client that shows a login
screen with no Vulkan frames in it: that is third-party behaviour a compositor
for games has to handle, and describing it reveals nothing about how we are put
together. The rule is about topology, not vocabulary.
Two comments still name the transport by its old name and are left for the
commit that renames it, so that rename reads as one change rather than as
noise spread across four imports.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wanjohi
DatCaptainHorse
KAAL1 (Bingus)
