github-mirrors/netris-nestri
1
0
Fork 0
mirror of https://github.com/nestriness/nestri.git synced 2026-09-19 17:25:19 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix(nescope): make HDR reachable — start XWayland, advertise the opaque FourCCs (#314)

Browse Source 
Three related fixes. Together they take HDR from unreachable to working
end to end on the XWayland path.

## XWayland was never started

Three lines had been commented out since the initial import: the call
that spawns XWayland, the guard that waits for it, and the `DISPLAY`
handed to the child. Every game therefore launched as a native Wayland
client. Nothing reported it -- the compositor still logged the X display
it was telling clients to point at, which is why it read as working.

That is also why HDR never fired. The colour space is signalled over a
protocol whose Vulkan layer lives in the game process and finds the
compositor through the X11 root window, so the one path able to carry it
was the one path no game was on. `ENABLE_GAMESCOPE_WSI` and `DXVK_HDR`
were already being set, which switched that layer on and then handed it
a
display it could not use.

Restoring the guard also fixes the ordering it was written for: the
launch
now happens after XWayland reports ready rather than ~40ms before it.

## Mesa was dropping every format we advertised alpha-only

Mesa tracks two flags per VkFormat -- one contributed by a format alpha
FourCC, one by its opaque FourCC -- and skips any format carrying only
one:

```c
if (!(disp_fmt->flags & WSI_WL_FMT_ALPHA) ||
   !(disp_fmt->flags & WSI_WL_FMT_OPAQUE))
   continue;
```

We advertised `ARGB8888` and `XRGB8888`, so `B8G8R8A8` survived and made
the list look like it was working. Everything else was alpha-only and
was
dropped in silence -- `ABGR8888` had been advertised all along while
`R8G8B8A8` never once appeared on a surface. Adding the opaque spellings
takes the surface from 6 formats to 21 and restores the three that carry
HDR.

The comment above that list claimed it was for XWayland DRI3 and that a
game swapchain format was independent of it. It was the opposite: the
list decides what a game can select, and deleting an entry removes that
format from every client.

## Verified against swapchains, not format lists

A client asking for `A2B10G10R10` + `HDR10_ST2084` now gets a swapchain
and the compositor is told colorspace `1000104008`; one asking for
`R16G16B16A16_SFLOAT` + scRGB linear gets `1000104002`. Previously both
were refused at creation -- the WSI layer re-checks the requested format
against the driver own surface list, so the colour space and the pixel
format arrive from two different places and only one was being supplied.

`apps/nescope/scripts/verify-hdr-formats.sh` asks what a client is
offered
from inside a child process, keeping the XCB and Wayland surfaces apart
since a game presents through the XCB one. The default mode guards both
halves of what the compositor controls; `--expect-layer` states the full
target and passes once a WSI layer is present. No new dependencies
(`vulkaninfo` + `python3`).

## Still open

HDR is XWayland-only, documented as a FIXME in `hdr.rs`. A WSI layer
binds
the swapchain factory on its own Wayland connection while a native
client
surface lives on the client one, and object IDs do not cross
connections.
The FIXME records the fix both reference implementations point at, and
the
trap to avoid when we take it: gating format injection on "the
compositor
supports HDR" rather than on being able to signal the surface hands a
client PQ pixels that arrive tagged as SDR, with nothing reporting an
error.

nescope ships no WSI layer of its own; the above was verified with the
stock gamescope one, which drives our protocol unmodified.















<!-- greptile_comment -->

<h3>Greptile Summary</h3>

This PR makes HDR-capable native Wayland presentation reachable, adds
the alpha/opaque DMA-BUF FourCC pairs Mesa requires, makes XWayland an
explicit compatibility mode, and adds an HDR surface-format diagnostic.
- Starts XWayland only with `--xwayland`, waits for readiness before
launching the child, and stops cleanly if startup fails or times out.
- Routes Proton through Wayland unconditionally while retaining
`DXVK_HDR` as an HDR-specific setting.
- Advertises paired alpha and opaque FourCC variants with portable
modifiers.
- Separates XCB and Wayland probe results, selects one hardware adapter,
and distinguishes probe failures from format regressions.
- Documents the limitations of the legacy gamescope WSI path and the
external-layer dependency.

<h3>Confidence Score: 5/5</h3>

The PR appears safe to merge; no outstanding correctness, security, or
repository-rule issue remains.

All previous findings are resolved in the current code, including the
XWayland failure lifecycle, removal of unsupported vendor modifiers,
corrected HDR documentation, reliable diagnostic failure handling,
per-GPU format selection, and unconditional Proton Wayland routing. The
changes since the previous review preserve diagnostic output handling
without introducing a new failure.

<h3>Important Files Changed</h3>




| Filename | Overview |
|----------|----------|
| apps/nescope/src/main.rs | Adds opt-in XWayland lifecycle handling,
readiness timeout, conditional DISPLAY propagation, and unconditional
Proton Wayland routing; the previous launch-environment finding is
fixed. |
| apps/nescope/src/state.rs | Stops the event loop on reported XWayland
startup failure and advertises portable paired alpha/opaque DMA-BUF
formats without vendor-specific modifiers. |
| apps/nescope/src/hdr.rs | Documents the working native Wayland HDR
path and accurately distinguishes it from the external, deliberately
disabled gamescope WSI route. |
| apps/nescope/scripts/verify-hdr-formats.sh | Adds a diagnostic that
keeps GPU and surface paths separate and now preserves the intended exit
behavior when filtered Vulkan diagnostics contain no matching lines. |


<h3>Flowchart</h3>

```mermaid
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Launch[nescope child launch] --> Mode{--xwayland?}
  Mode -->|No| Wayland[Native Wayland surface]
  Mode -->|Yes| Wait[Start and await XWayland]
  Wait -->|Ready| XCB[XCB / XWayland surface]
  Wait -->|Error or 10s timeout| Stop[Log failure and stop]
  Wayland --> Formats[Paired alpha and opaque FourCCs]
  Formats --> HDR[HDR10 and scRGB formats available]
  XCB --> SDR[X11 compatibility path without native HDR]
  Proton[Proton child] -->|PROTON_ENABLE_WAYLAND=1| Wayland
```

<sub>Reviews (9): Last reviewed commit: ["nescope/scripts: guard the
diagnostic
pi..."](f8bdd68f87)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=60368324)</sub>

**Context used:**

- Knowledge Base — [Compositor, display, and input
control](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/compositor-input.md)
- Knowledge Base — [Streaming host
runtime](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/streaming-runtime.md)

<!-- /greptile_comment -->
This commit is contained in:
KAAL1 (Bingus)
2026-09-04 19:18:46 +03:00
committed by GitHub
parent 200bc9c75f
commit e70f05e245
4 changed files with 537 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

278
apps/nescope/scripts/verify-hdr-formats.sh Executable file
View File

@@ -0,0 +1,278 @@
#!/usr/bin/env bash
# Report the surface formats a client actually sees under nescope.
#
# HDR reaches a game as a (VkFormat, VkColorSpaceKHR) pair on its swapchain
# surface. Everything else -- the colour-management protocol, the dmabuf format
# list, the encoder's matrix and transfer tags -- is downstream of whether that
# pair was ever offered. So this asks the one question directly, from inside a
# real client process, using the loader's own enumeration rather than ours.
#
# It deliberately does not check pixels. verify-hdr.sh in nescapture does that,
# and the two are answering different questions: this one is "was HDR on the
# menu", that one is "did the samples come out where the standard says". A pass
# here and a fail there means we converted wrongly; a fail here means the game
# never had the option and anything downstream is moot.
#
# The two surfaces a client can be on need separate answers, so there are two
# modes:
#
# (default) The Wayland surface, which is where HDR actually comes
# from: Mesa pairs the colour spaces it learns from
# wp_color_manager_v1 with the pixel formats it derives from
# the dmabuf list. Guards both halves. Each is silently
# absent when broken, and the colour space alone is what made
# an 8-bit surface look like working HDR.
#
# --expect-layer The XCB surface, for diagnosing the legacy route only.
# Mesa offers no HDR colour space on XWayland at all, so the
# HDR pairs can only come from a WSI layer inside the game's
# process -- gamescope's approach, which predates Wayland
# colour management. nescope does not ship such a layer and
# does not enable one, because capture cannot see the colour
# space it hides and would tag PQ samples as BT.709. This
# mode passes only with a layer loaded, e.g.
#
# VK_ADD_IMPLICIT_LAYER_PATH=<dir> \
# apps/nescope/scripts/verify-hdr-formats.sh --expect-layer
#
# A failure here is the expected state, not a regression.
#
# Exit codes are distinct on purpose: 0 pass, 1 the formats are wrong, 2 the
# environment or the probe failed and this run measured nothing.
#
# Usage: apps/nescope/scripts/verify-hdr-formats.sh [--expect-layer]
set -euo pipefail
MODE="baseline"
NESCOPE_ARGS=()
if [ "${1:-}" = "--expect-layer" ]; then
MODE="layer"
# nescope no longer sets this, on purpose, so the legacy layer stays inert in
# normal use. This mode is the one place that wants it, so it opts in here
# rather than relying on the compositor to leave a switch flipped.
export ENABLE_GAMESCOPE_WSI=1
# And the route only exists on the XCB surface, which needs a server.
NESCOPE_ARGS=(--xwayland)
fi
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
: "${XDG_RUNTIME_DIR:=/run/user/$(id -u)}"
export XDG_RUNTIME_DIR
for tool in vulkaninfo python3; do
command -v "$tool" >/dev/null || { echo "missing required tool: $tool" >&2; exit 1; }
done
echo "building…"
cargo build --release -p nescope --manifest-path "$ROOT/Cargo.toml" >/dev/null
# The probe runs as nescope's child, so it sees exactly the environment a game
# would. Which display variables it inherits is itself a result -- a child with
# no DISPLAY is a native Wayland client, and that is the path with no HDR -- so
# record them before enumerating anything.
cat > "$WORK/probe.sh" <<'PROBE'
#!/usr/bin/env bash
echo "child_wayland_display=${WAYLAND_DISPLAY:-unset}"
echo "child_display=${DISPLAY:-unset}"
echo "child_enable_gamescope_wsi=${ENABLE_GAMESCOPE_WSI:-unset}"
echo "---VULKANINFO---"
# Keep stderr. A Vulkan init failure and a missing HDR format both end up as
# "no formats" otherwise, and only one of them is this script's business.
if vulkaninfo 2>&1; then
echo "probe_vulkaninfo=ok"
else
echo "probe_vulkaninfo=failed rc=$?"
fi
PROBE
chmod +x "$WORK/probe.sh"
echo "enumerating surface formats under nescope…"
NESCOPE_RC=0
timeout 60 "$ROOT/target/release/nescope" --hdr "${NESCOPE_ARGS[@]}" \
--width 1280 --height 720 \
-- "$WORK/probe.sh" > "$WORK/probe.out" 2>"$WORK/nescope.log" || NESCOPE_RC=$?
# A compositor that died, or a probe that never got a Vulkan instance, is an
# environment failure. Reporting it as missing HDR formats would be the exact
# confusion this script exists to prevent, so say which one it was and show the
# log rather than deleting it with the temp dir.
if [ "$NESCOPE_RC" -eq 124 ]; then
echo
echo "FAIL — nescope did not exit within 60s; the run was killed." >&2
echo "This is an environment failure, not an HDR result. Last log lines:" >&2
tail -20 "$WORK/nescope.log" >&2
exit 2
fi
if grep -q "^probe_vulkaninfo=failed" "$WORK/probe.out"; then
echo
echo "FAIL — vulkaninfo failed inside the compositor." >&2
echo "This is an environment failure, not an HDR result:" >&2
sed -n "/---VULKANINFO---/,/probe_vulkaninfo=failed/p" "$WORK/probe.out" \
| grep -iE "error|cannot|failed|no such" | head -10 >&2 || true
exit 2
fi
if ! grep -q "^probe_vulkaninfo=ok" "$WORK/probe.out"; then
echo
echo "FAIL — the probe did not run to completion (nescope rc=$NESCOPE_RC)." >&2
echo "This is an environment failure, not an HDR result. Last log lines:" >&2
tail -20 "$WORK/nescope.log" >&2
exit 2
fi
python3 - "$WORK/probe.out" "$MODE" <<'PY'
import re, sys
path, mode = sys.argv[1], sys.argv[2]
text = open(path, errors="replace").read()
env = dict(re.findall(r"^(child_\w+)=(.*)$", text, re.M))
print()
print(f"child WAYLAND_DISPLAY: {env.get('child_wayland_display', '?')}")
print(f"child DISPLAY: {env.get('child_display', '?')}")
print(f"ENABLE_GAMESCOPE_WSI: {env.get('child_enable_gamescope_wsi', '?')}")
# Pull the format lists from the presentable-surface section of the first real
# GPU. llvmpipe is enumerated too and would double every count, so skip any
# adapter that names it -- a software rasteriser's opinion about HDR is not the
# thing under test.
#
# Keep the XCB and Wayland surfaces apart. Once the child has a DISPLAY it has
# both, they carry different formats, and merging them would let one path's HDR
# support stand in for the other's. The XCB list is the one a Proton game sees.
# One adapter only, and named in the output. Appending every adapter's formats
# into one list would let a second GPU satisfy the checks while the one the game
# runs on lacks the formats entirely -- a pass that means nothing. There is no
# way to ask vulkaninfo which adapter a game would pick, so this takes the first
# hardware one and says which, leaving a multi-GPU host to be read rather than
# guessed at.
section = text.split("Presentable Surfaces", 1)
by_path = {"xcb": [], "wayland": []}
gpu, path, chosen_gpu, other_gpus = None, None, None, []
if len(section) > 1:
block, cur_fmt = section[1], None
for line in block.splitlines():
m = re.match(r"\s*GPU id\s*:\s*\d+\s*\((.+?)\)\s*\[(.+?)\]", line)
if m:
gpu, exts = m.group(1), m.group(2)
path = "wayland" if "wayland_surface" in exts else "xcb"
if "llvmpipe" not in gpu:
if chosen_gpu is None:
chosen_gpu = gpu
elif gpu != chosen_gpu and gpu not in other_gpus:
other_gpus.append(gpu)
continue
# Skip the software rasteriser, and every hardware adapter after the
# first: their formats are not the ones under test.
if gpu is None or "llvmpipe" in gpu or gpu != chosen_gpu:
continue
m = re.match(r"\s*format\s*=\s*(\S+)", line)
if m:
cur_fmt = m.group(1)
continue
m = re.match(r"\s*colorSpace\s*=\s*(\S+)", line)
if m and cur_fmt and path:
# vulkaninfo pads its listing with FORMAT_UNDEFINED entries when a
# layer appends to the surface format list. Checked against the API
# directly with the two-call pattern -- the count and the entries
# agree there, and VK_INCOMPLETE comes back on a short buffer -- so
# these are an artifact of the listing, not formats a client sees.
if cur_fmt != "FORMAT_UNDEFINED":
by_path[path].append((cur_fmt, m.group(1)))
cur_fmt = None
# A game under XWayland presents through the XCB surface, so that is the list
# under test whenever it exists. Fall back to the Wayland one when the child
# never got a DISPLAY, which is the only case where it is what a game would use.
on_xwayland = bool(by_path["xcb"])
formats = by_path["xcb"] if on_xwayland else by_path["wayland"]
fails = []
if not formats:
fails.append("no surface formats enumerated at all — the probe never reached "
"a surface, so this run measured nothing")
print(f"\nadapter under test: {chosen_gpu or '(none found)'}")
if other_gpus:
print(f" ignoring {len(other_gpus)} other adapter(s): {', '.join(other_gpus)}")
print(" a game may not pick the one above; check it is the render device")
print(f"path under test: {'XCB (XWayland)' if on_xwayland else 'native Wayland'}")
if on_xwayland:
print(f" (native Wayland surface offers {len(by_path['wayland'])} formats, not under test)")
print(f"\nsurface formats offered: {len(formats)}")
for f, cs in formats:
print(f" {f:<34} {cs}")
spaces = {cs for _, cs in formats}
depth10 = [f for f, _ in formats if "10" in f and "B8G8R8A8" not in f]
depth16 = [f for f, _ in formats if "16G16" in f or "SFLOAT" in f]
print()
print(f"HDR10_ST2084 offered: {'yes' if any('ST2084' in s for s in spaces) else 'no'}")
print(f"scRGB linear offered: {'yes' if any('EXTENDED_SRGB_LINEAR' in s for s in spaces) else 'no'}")
print(f"10-bit formats offered: {'yes' if depth10 else 'no'}")
print(f"FP16 formats offered: {'yes' if depth16 else 'no'}")
if formats:
if mode == "baseline":
# Only what is genuinely working today. The point of this mode is to
# notice if the colour-management path stops being wired up at all,
# which would otherwise look identical to plain SDR.
# Guard both halves of what the compositor itself controls: the colour
# spaces, which come from the colour-management protocol, and the pixel
# formats, which come from the dmabuf list. Each fails silently on its
# own -- a missing format is simply absent, with nothing logged.
wl = by_path["wayland"]
wl_spaces = {cs for _, cs in wl}
if not any("ST2084" in s for s in wl_spaces):
fails.append("HDR10_ST2084_EXT is no longer offered on the Wayland "
"surface — the wp_color_manager_v1 path has stopped "
"being advertised")
if not [f for f, _ in wl if "10" in f and "B8G8R8A8" not in f]:
fails.append("no 10-bit pixel format on the Wayland surface — Mesa "
"drops any format lacking either its alpha or its "
"opaque FourCC spelling, so check both are advertised")
if not [f for f, _ in wl if "16G16" in f]:
fails.append("no FP16 pixel format on the Wayland surface — the "
"scRGB path needs R16G16B16A16_SFLOAT")
else:
# The three a WSI layer injects. Until one exists these all fail, and
# that is the expected reading, not a defect in this script.
if not any("ST2084" in s for s in spaces):
fails.append("HDR10_ST2084_EXT not offered")
if not depth10:
fails.append("no 10-bit format offered — PQ over 8-bit BGRA bands; a "
"WSI layer must inject A2B10G10R10/A2R10G10B10")
if not any("EXTENDED_SRGB_LINEAR" in s for s in spaces):
fails.append("EXTENDED_SRGB_LINEAR_EXT not offered — Proton titles on "
"the scRGB path find no matching format and fall back to SDR")
if not depth16:
fails.append("no FP16 format offered — the scRGB path needs "
"R16G16B16A16_SFLOAT")
# The display the child inherited decides which of the two code paths it is on,
# and only one of them can carry HDR today. Worth saying plainly either way,
# because a native-Wayland client failing the layer checks above is failing for
# a reason that has nothing to do with formats.
if env.get("child_display", "unset") == "unset":
print("\nnote: the child had no DISPLAY and ran as a native Wayland client,")
print(" which is the intended configuration — HDR is only offered on the")
print(" Wayland surface. Pass --xwayland to nescope for X11-only software,")
print(" at the cost of that software's HDR.")
else:
print("\nnote: the child had a DISPLAY, so XWayland is running. A game that")
print(" presents through it gets no HDR: Mesa offers no HDR colour space")
print(" on the XWayland surface.")
print()
if fails:
print("FAIL" + (" (expected until the WSI layer lands)" if mode == "layer" else ""))
for f in fails:
print(f" - {f}")
sys.exit(1)
print("PASS")
PY

86
apps/nescope/src/hdr.rs
View File

@@ -1,18 +1,82 @@
//! HDR / color management protocol handlers. //! HDR / colour management protocol handlers.
//! //!
//! Two signalling paths feed into [`HdrState`]: //! # HDR here is Wayland colour management
//! //!
//! 1. **`wp_color_management_v1`** — the standard staging Wayland protocol. //! A game gets HDR by being a Wayland client and asking for it through
//! Wine/Proton/SDL2 uses this when the game requests an HDR swapchain via //! `wp_color_manager_v1`, which Mesa turns into HDR colour spaces on the
//! standard Vulkan color-space extensions. //! surface. That is the whole mechanism, it works, and it needs nothing outside
//! this tree.
//! //!
//! 2. **`gamescope_swapchain_factory_v2`** — Valve's private protocol used by //! Measured on RDNA4, and the asymmetry is the point:
//! the gamescope WSI Vulkan layer. This is the primary path for Steam
//! games using PROTON_ENABLE_NVAPI / HDR10_ST2084.
//! //!
//! nescope never performs color conversion itself — it just tracks which color //! | surface | formats offered | HDR |
//! space the active surface has declared so that an external capture library //! |---|---|---|
//! (e.g. the Vulkan vkcapture layer) can retrieve it via the public API. //! | Wayland | 21, incl. `A2B10G10R10` and `R16G16B16A16_SFLOAT` | yes |
//! | XWayland | 2, both 8-bit sRGB | no -- "surface offers no format in HDR10_ST2084_EXT" |
//!
//! Verified past the swapchain too, not just on the format list: a client
//! requesting `A2B10G10R10` + `HDR10_ST2084` comes out `yuv420p10le`, full
//! range, `bt2020nc` / `smpte2084` / `bt2020`.
//!
//! So the launch environment always sets `PROTON_ENABLE_WAYLAND=1` -- not as an
//! HDR switch but as the way a Windows title reaches the compositor at all,
//! since Proton renders through XWayland otherwise and XWayland is off by
//! default. `--hdr` adds `DXVK_HDR=1` (DXVK's dxgi gates HDR exposure on it).
//! Those two are what HDR needs.
//!
//! Mesa pairs the colour spaces it learns here with the pixel formats it
//! derives from our `zwp_linux_dmabuf_v1` list, so both halves have to be
//! present -- the surface offered nothing but `B8G8R8A8` until the format list
//! advertised the opaque FourCC spellings alongside the alpha ones. See the
//! list in `state.rs`, which is where that constraint lives.
//!
//! # `gamescope_swapchain_factory_v2` is the legacy route, and stays off
//!
//! Also implemented here, because it costs little and a host may deliberately
//! want it. It predates Wayland colour management and works the other way
//! round: a WSI layer inside the game's process appends HDR colour spaces Mesa
//! never offered, rewrites `imageColorSpace` to `SRGB_NONLINEAR` so the driver
//! is never told HDR is happening, and reports the real colour space to the
//! compositor over this protocol instead.
//!
//! It is not how we do HDR, for three reasons that all point the same way:
//!
//! - It needs a Vulkan layer this tree does not ship. Verified working with
//! gamescope's own, unmodified -- the XML here is byte-identical to theirs,
//! and the atoms written in `state.rs` are what it reads.
//! - It only helps the XWayland path, which is the path without HDR anyway.
//! - **Capture reads the colour space it hides.** A game asking for HDR10
//! through it has its ten-bit PQ samples encoded and tagged BT.709 SDR, at
//! full frame rate, decoding cleanly. Recorded where that value is read, in
//! the capture layer's swapchain hook.
//!
//! That last one makes enabling it worse than leaving it off: it trades no HDR
//! for wrong HDR. So `GAMESCOPE_WAYLAND_DISPLAY` is set for the child but
//! `ENABLE_GAMESCOPE_WSI` deliberately is not, which leaves the layer inert
//! unless someone opts in. If anyone ever does want this path, the colour space
//! it reports arrives here and the capture layer cannot see it, so it would
//! need a channel from this process to that one.
//!
//! # What HDR does not cover
//!
//! A game that cannot be a Wayland client gets SDR, and XWayland is off by
//! default -- it costs input latency and a compositing hop, which is the wrong
//! trade for a streaming box, and Proton does not need it. `--xwayland` turns
//! it on for the shrinking set of X11-only native software, which then runs
//! without HDR: Mesa offers no HDR colour space on the XWayland surface, and
//! nothing in this module can change that.
//!
//! Still unexercised: no game has run, and the scRGB/FP16 arm has had no pixels
//! through it -- only HDR10 PQ.
//!
//! # Signalling paths, for reference
//!
//! Both feed [`HdrState`], which tracks the colour space the active surface has
//! declared. This module never converts anything itself.
//!
//! 1. **`wp_color_manager_v1`** -- the standard protocol, and the live one.
//! 2. **`gamescope_swapchain_factory_v2`** -- the legacy route described above,
//! reachable only if a WSI layer is present and opted into.
#![allow(unused)] #![allow(unused)]
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Mutex; use std::sync::Mutex;

184
apps/nescope/src/main.rs
View File

@@ -25,7 +25,8 @@
//! | Variable | Effect | //! | Variable | Effect |
//! |----------------|-----------------------------------------------| //! |----------------|-----------------------------------------------|
//! | `WAYLAND_DISPLAY` | Set by nescope before spawning the game | //! | `WAYLAND_DISPLAY` | Set by nescope before spawning the game |
//! | `DISPLAY` | Set to the XWayland display (`:N`) | //! | `DISPLAY` | XWayland display (`:N`), only with `--xwayland` |
//! | `PROTON_ENABLE_WAYLAND` | Set to `1` always, so Proton renders through Wayland |
//! | `XCURSOR_THEME` | XCursor theme name for the software cursor | //! | `XCURSOR_THEME` | XCursor theme name for the software cursor |
//! | `XCURSOR_SIZE` | XCursor size in pixels | //! | `XCURSOR_SIZE` | XCursor size in pixels |
//! | `RUST_LOG` | Tracing filter (e.g. `nescope=debug`) | //! | `RUST_LOG` | Tracing filter (e.g. `nescope=debug`) |
@@ -45,6 +46,11 @@ use std::os::unix::process::CommandExt;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
/// How long to wait for XWayland to report a display before giving up. Startup
/// is normally tens of milliseconds; this only has to be longer than a slow
/// machine's worst case, not tuned.
const XWAYLAND_TIMEOUT_SECS: u64 = 10;
use calloop::generic::Generic; use calloop::generic::Generic;
use calloop::signals::{Signal, Signals}; use calloop::signals::{Signal, Signals};
use calloop::timer::Timer; use calloop::timer::Timer;
@@ -96,6 +102,17 @@ struct Args {
#[arg(long, env = "NESCOPE_HDR")] #[arg(long, env = "NESCOPE_HDR")]
hdr: bool, hdr: bool,
/// Run XWayland, for Linux-native software with no Wayland support.
///
/// Off by default, and that is the point. XWayland costs input latency and
/// a compositing hop, which is the wrong trade for a streaming box. Windows
/// titles do not need it -- Proton renders through Wayland when told to,
/// which is what the launch environment does -- and HDR is only offered on
/// the Wayland surface, so a game routed through XWayland loses it too.
/// Turn this on for the shrinking set of X11-only native software.
#[arg(long, env = "NESCOPE_XWAYLAND")]
xwayland: bool,
/// Wayland socket name (created in $XDG_RUNTIME_DIR). /// Wayland socket name (created in $XDG_RUNTIME_DIR).
#[arg(long, default_value = "nescope-0", env = "NESCOPE_SOCKET")] #[arg(long, default_value = "nescope-0", env = "NESCOPE_SOCKET")]
socket: String, socket: String,
@@ -278,7 +295,9 @@ fn main() {
args.hdr, args.hdr,
args.render_device.clone(), args.render_device.clone(),
); );
//state.init_xwayland(&loop_handle, Some(args.x_display)); if args.xwayland {
state.init_xwayland(&loop_handle, Some(args.x_display));
}
// Said out loud because in compositor mode nothing else can work them out. // Said out loud because in compositor mode nothing else can work them out.
// A process started by the hub rather than by nescope has no inherited // A process started by the hub rather than by nescope has no inherited
@@ -286,7 +305,11 @@ fn main() {
if args.command.is_empty() { if args.command.is_empty() {
tracing::info!( tracing::info!(
wayland_display = %socket_name.to_string_lossy(), wayland_display = %socket_name.to_string_lossy(),
display = format!(":{}", args.x_display), display = if args.xwayland {
format!(":{}", args.x_display)
} else {
"(none — XWayland off; pass --xwayland if you need it)".to_string()
},
"compositor mode — point clients at these and they will connect" "compositor mode — point clients at these and they will connect"
); );
} }
@@ -402,6 +425,11 @@ fn main() {
// Run with a 1-second timeout so the idle closure fires even when no // Run with a 1-second timeout so the idle closure fires even when no
// Wayland events arrive (needed for zombie reaping and auto-exit checks). // Wayland events arrive (needed for zombie reaping and auto-exit checks).
// Deadline for XWayland to come up. The launch below waits on it, so if it
// never arrives there is nothing to wait for and no game to run.
let startup = std::time::Instant::now();
let mut xwayland_timed_out = false;
event_loop event_loop
.run(Some(Duration::from_secs(1)), &mut data, move |data| { .run(Some(Duration::from_secs(1)), &mut data, move |data| {
// ── Reap zombie children ────────────────────────────────── // ── Reap zombie children ──────────────────────────────────
@@ -415,62 +443,110 @@ fn main() {
&& data.primary_pid.is_none() && data.primary_pid.is_none()
&& !data.state.game_launched && !data.state.game_launched
{ {
//if let Some(xdisplay) = data.state.xdisplay { // Only wait on XWayland when we are the ones providing it.
data.state.game_launched = true; // Without --xwayland there is no display coming, so waiting
tracing::info!("Launching {:?}", command[0]); // would mean never launching.
if !args.xwayland || data.state.xdisplay.is_some() {
data.state.game_launched = true;
tracing::info!("Launching {:?}", command[0]);
let mut cmd = std::process::Command::new(&command[0]); let mut cmd = std::process::Command::new(&command[0]);
cmd.args(&command[1..]) cmd.args(&command[1..])
//.env("DISPLAY", format!(":{xdisplay}")) .stdin(std::process::Stdio::null())
.stdin(std::process::Stdio::null()) .stdout(std::process::Stdio::inherit())
.stdout(std::process::Stdio::inherit()) .stderr(std::process::Stdio::inherit())
.stderr(std::process::Stdio::inherit()) // Put the game in its own process group so we can
// Put the game in its own process group so we can // kill the whole tree at once with kill(-pgid, …).
// kill the whole tree at once with kill(-pgid, …). .process_group(0)
.process_group(0) .env("WAYLAND_DISPLAY", &gamescope_wayland_socket);
// Provide also WAYLAND_DISPLAY, so if the game or application
// is Wayland-native and doesn't support older X11 it'll still run.
.env("WAYLAND_DISPLAY", &gamescope_wayland_socket);
if args.hdr { // DISPLAY only if XWayland is actually running. Setting it
tracing::debug!( // otherwise points clients at a server that is not there,
gamescope_wayland_socket, // which is what the compositor used to do.
"Setting GAMESCOPE_WAYLAND_DISPLAY for application" if let Some(xdisplay) = data.state.xdisplay {
cmd.env("DISPLAY", format!(":{xdisplay}"));
}
// Proton renders through XWayland unless this is set, and
// XWayland is off by default -- so without this a Windows
// title has no display at all. Unconditional for that
// reason: it is how the game reaches the compositor, not
// an HDR switch. It is also what makes HDR reachable, since
// colour management only exists on the Wayland surface --
// measured here, that surface offers 21 formats including
// HDR10 over A2B10G10R10 while the XWayland one offers two,
// both 8-bit sRGB.
cmd.env("PROTON_ENABLE_WAYLAND", "1");
if args.hdr {
// DXVK's dxgi.dll gates HDR colour space exposure on
// this. Without it neither DX11 nor DX12 (vkd3d-proton
// through DXVK's dxgi) sees HDR as available.
cmd.env("DXVK_HDR", "1");
// Left set, but deliberately without ENABLE_GAMESCOPE_WSI
// alongside it, so it is inert unless somebody opts in.
//
// That pair activates gamescope's WSI layer, which
// predates Wayland colour management and works by
// hiding HDR from the driver and reporting it to the
// compositor out of band. We do not want it: it needs a
// layer this image does not ship, it only helps the
// XWayland path, and capture reads the colour space it
// hides -- measured, a game asking for HDR10 through it
// has its PQ samples encoded and tagged BT.709 SDR.
// Enabling it would trade no HDR for wrong HDR.
tracing::debug!(
gamescope_wayland_socket,
"HDR: Wayland colour management; gamescope WSI not enabled"
);
cmd.env("GAMESCOPE_WAYLAND_DISPLAY", &gamescope_wayland_socket);
}
// Detect GPU vendor from render device and set VK_DRIVER_FILES
// so the game uses the same GPU as nescope.
if let Some(ref rd) = args.render_device {
if let Some(icd_path) = detect_gpu_icd(rd) {
cmd.env("VK_ICD_FILENAMES", &icd_path);
cmd.env("VK_DRIVER_FILES", &icd_path); // Mesa fallback
tracing::info!("GPU ICD → {icd_path}");
}
}
match cmd.spawn() {
Ok(child) => {
let pid = child.id();
tracing::info!("Game process spawned (pid {pid})");
data.primary_pid = Some(pid as i32);
data.game_pgid = Some(pid as i32); // PGID == PID due to .process_group(0)
data.game_process = Some(child);
}
Err(e) => {
tracing::error!("Failed to launch {:?}: {e}", command[0]);
data.loop_signal.stop();
return;
}
}
} else if args.xwayland
&& !xwayland_timed_out
&& startup.elapsed() > Duration::from_secs(XWAYLAND_TIMEOUT_SECS)
{
// Waiting forever is the outcome to avoid: the auto-exit
// below only runs once a game has been launched, so a
// display that never arrives leaves nescope polling with no
// game and nothing logged. Smithay does not always report a
// failed XWayland as an error -- an Xwayland that exits
// immediately simply never becomes ready -- so this is a
// deadline, not an error handler.
xwayland_timed_out = true;
tracing::error!(
"XWayland did not become ready within {XWAYLAND_TIMEOUT_SECS}s — \
cannot launch a game without a display"
); );
cmd.env("GAMESCOPE_WAYLAND_DISPLAY", &gamescope_wayland_socket); kill_all_children();
cmd.env("ENABLE_GAMESCOPE_WSI", "1"); data.loop_signal.stop();
// DXVK's dxgi.dll gates HDR color space exposure on this env var.
// Without it, both DX11 (DXVK) and DX12 (vkd3d-proton via DXVK dxgi)
// games will not see HDR as available.
cmd.env("DXVK_HDR", "1");
} }
// Detect GPU vendor from render device and set VK_DRIVER_FILES
// so the game uses the same GPU as nescope.
if let Some(ref rd) = args.render_device {
if let Some(icd_path) = detect_gpu_icd(rd) {
cmd.env("VK_ICD_FILENAMES", &icd_path);
cmd.env("VK_DRIVER_FILES", &icd_path); // Mesa fallback
tracing::info!("GPU ICD → {icd_path}");
}
}
match cmd.spawn() {
Ok(child) => {
let pid = child.id();
tracing::info!("Game process spawned (pid {pid})");
data.primary_pid = Some(pid as i32);
data.game_pgid = Some(pid as i32); // PGID == PID due to .process_group(0)
data.game_process = Some(child);
}
Err(e) => {
tracing::error!("Failed to launch {:?}: {e}", command[0]);
data.loop_signal.stop();
return;
}
}
//}
} }
// ── Poll primary process ────────────────────────────────── // ── Poll primary process ──────────────────────────────────

87
apps/nescope/src/state.rs
View File

@@ -370,7 +370,13 @@ impl NescopeState {
NescopeState::open_x11_input_conn(data, display_number); NescopeState::open_x11_input_conn(data, display_number);
} }
XWaylandEvent::Error => { XWaylandEvent::Error => {
tracing::error!("XWayland crashed at startup"); // The game launch waits for a display number, so a dead
// XWayland means it will never start. Nothing downstream can
// notice that: the auto-exit path keys off a game having been
// launched, so without stopping here nescope would poll
// forever with no game and no reason given.
tracing::error!("XWayland crashed at startup — cannot run a game without it");
data.loop_signal.stop();
} }
}); });
@@ -934,38 +940,53 @@ where
use smithay::wayland::dmabuf::{DmabufFeedbackBuilder, DmabufState}; use smithay::wayland::dmabuf::{DmabufFeedbackBuilder, DmabufState};
use std::os::unix::fs::MetadataExt; use std::os::unix::fs::MetadataExt;
// Formats declared to XWayland for DRI3. The actual pixel format used // Formats offered to clients through zwp_linux_dmabuf_v1. This list does
// by the game's Vulkan swapchain is independent of this list. // decide what pixel formats a game's Vulkan swapchain can use: Mesa's
let formats = [ // Wayland WSI derives its surface formats from it, so a format missing here
Format { // is a format no client can select.
code: Fourcc::Argb8888, //
modifier: Modifier::Linear, // Each VkFormat needs BOTH its alpha and its opaque FourCC spelling.
}, // Mesa tracks those as two flags on one VkFormat -- ARGB8888 contributes
Format { // the alpha flag, XRGB8888 the opaque one -- and skips any format that does
code: Fourcc::Xrgb8888, // not carry both, so advertising only the alpha variant silently drops it.
modifier: Modifier::Linear, // That is a quiet failure: the format simply never appears, with nothing
}, // logged at either end.
Format { //
code: Fourcc::Abgr8888, // The 10-bit and FP16 pairs are what carry HDR. A game asks for HDR through
modifier: Modifier::Linear, // a WSI layer that injects the HDR colour spaces, but the layer re-checks
}, // the requested VkFormat against the driver's own surface list and refuses
Format { // the swapchain outright if it is absent. So the colour space and the pixel
code: Fourcc::Abgr2101010, // format come from two different places, and HDR needs both.
modifier: Modifier::Linear, let formats = {
}, use std::iter::once;
Format {
code: Fourcc::Argb2101010, // Only LINEAR and INVALID, deliberately. Naming a driver's tiled
modifier: Modifier::Linear, // modifiers here would mean hard-coding one vendor's values into a list
}, // sent to every client, and it buys nothing: INVALID leaves the choice
Format { // of tiling to the driver, which picks its own optimal layout, and
code: Fourcc::Argb8888, // LINEAR is universally supported as the fallback. Measured on RDNA4 --
modifier: Modifier::Invalid, // adding that vendor's eight tiled modifiers changes neither the
}, // formats a client is offered nor whether an HDR swapchain is created.
Format { //
code: Fourcc::Xrgb8888, // alpha spelling, opaque spelling
modifier: Modifier::Invalid, const PAIRS: &[(Fourcc, Fourcc)] = &[
}, (Fourcc::Argb8888, Fourcc::Xrgb8888),
]; (Fourcc::Abgr8888, Fourcc::Xbgr8888),
(Fourcc::Abgr2101010, Fourcc::Xbgr2101010),
(Fourcc::Argb2101010, Fourcc::Xrgb2101010),
(Fourcc::Abgr16161616f, Fourcc::Xbgr16161616f),
];
PAIRS
.iter()
.flat_map(|&(alpha, opaque)| once(alpha).chain(once(opaque)))
.flat_map(|code| {
once(Modifier::Linear)
.chain(once(Modifier::Invalid))
.map(move |modifier| Format { code, modifier })
})
.collect::<Vec<_>>()
};
let mut dmabuf_state = DmabufState::new(); let mut dmabuf_state = DmabufState::new();