github-mirrors/netris-nestri
1
0
Fork 0
mirror of https://github.com/nestriness/nestri.git synced 2026-09-27 04:52:25 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: media bitrate control, HDR (#346)

Browse Source 
Fixes: #335 

Still a work-in-progress.

---------

Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Wanjohi <elviswanjohi47@gmail.com>
This commit is contained in:
Kristian Ollikainen
2026-09-25 12:13:34 +03:00
committed by GitHub
co-authored by DatCaptainHorse Claude Opus 5 Wanjohi
parent 1c721962f4
commit 0811f57f1a
64 changed files with 15151 additions and 2702 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+59 -68
View File
@@ -52,12 +52,6 @@ dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "android_system_properties"
version = "0.1.6"
@@ -341,7 +335,7 @@ checksum = "6a1f896587b6f2c069c73d2f0913e2d590c3990285cd2f0b6aa02b786b4c679c"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -359,16 +353,16 @@ dependencies = [
"bitflags 2.13.2",
"nix",
"polling",
"rustix 1.1.4",
"rustix 1.1.5",
"slab",
"tracing",
]
[[package]]
name = "cc"
version = "1.4.6"
version = "1.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3eb0f42d6c360dc3f8a821f6bf2fdea7f72bfd36b3076eb0e6d1e9e0752fff4"
checksum = "54413ede23c2daf518f35156dfde027feb2374004d63bd497f983c8db9c0e313"
dependencies = [
"find-msvc-tools",
"shlex 2.0.1",
@@ -401,9 +395,9 @@ dependencies = [
[[package]]
name = "cfg-if"
version = "1.0.4"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
[[package]]
name = "cfg_aliases"
@@ -496,7 +490,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -765,7 +759,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c54e03a951783e8b327515db3f2a2fd0e3bed362a96b066f341ce66ed49b4ead"
dependencies = [
"data-encoding",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -885,7 +879,7 @@ checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -926,7 +920,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51a91c9b32ac4e8105dec255e849e0d66e27d7c34d184364fb93e469db08f690"
dependencies = [
"drm-sys",
"rustix 1.1.4",
"rustix 1.1.5",
]
[[package]]
@@ -1013,7 +1007,7 @@ checksum = "0590c4a94da3372e83493b956755a6e2266830b6e4e3b101afe66e3f39477b91"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -1069,9 +1063,9 @@ checksum = "64cd1e32ddd350061ae6edb1b082d7c54915b5c672c389143b9a63403a109f24"
[[package]]
name = "find-msvc-tools"
version = "0.1.12"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d"
checksum = "ef25905e51abafe4dcea6c15fec58c57b601cdbd0ee53d22ea1d3016c587d39b"
[[package]]
name = "flate2"
@@ -1187,7 +1181,7 @@ checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -1221,9 +1215,9 @@ dependencies = [
[[package]]
name = "generator"
version = "0.8.9"
version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3b854b0e584ead1a33f18b2fcad7cf7be18b3875c78816b753639aa501513ae"
checksum = "54ade96dc9003043bce7c035c85a9df5a858bfb2039c5a2e6fdf00f324f6c551"
dependencies = [
"cc",
"cfg-if",
@@ -1250,7 +1244,7 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bd49230192a3797a9a4d6abe9b3eed6f7fa4c8a8a4947977c6f80025f92cbd8"
dependencies = [
"rustix 1.1.4",
"rustix 1.1.5",
"windows-link",
]
@@ -1352,8 +1346,6 @@ version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
@@ -1453,9 +1445,9 @@ dependencies = [
[[package]]
name = "hyper-rustls"
version = "0.27.9"
version = "0.27.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33ca68d021ef39cf6463ab54c1d0f5daf03377b70561305bb89a8f83aab66e0f"
checksum = "dfa8e654703247911e29c23fbeaa261834bd9bb74efba2f9acddc37bfb127f53"
dependencies = [
"http",
"hyper",
@@ -1637,7 +1629,7 @@ dependencies = [
"hyper",
"hyper-util",
"log",
"rand 0.10.2",
"rand 0.10.3",
"tokio",
"url",
"xmltree",
@@ -1744,7 +1736,7 @@ dependencies = [
"pin-project",
"portable-atomic",
"portmapper",
"rand 0.10.2",
"rand 0.10.3",
"reqwest",
"rustc-hash",
"rustls",
@@ -1774,7 +1766,7 @@ dependencies = [
"ed25519-dalek",
"getrandom 0.4.3",
"n0-error",
"rand 0.10.2",
"rand 0.10.3",
"serde",
"url",
"zeroize",
@@ -1794,7 +1786,7 @@ dependencies = [
"n0-error",
"n0-future",
"portable-atomic",
"rand 0.10.2",
"rand 0.10.3",
"rustls",
"simple-dns",
"strum",
@@ -1857,7 +1849,7 @@ dependencies = [
"num_enum",
"pin-project",
"postcard",
"rand 0.10.2",
"rand 0.10.3",
"reqwest",
"rustls",
"rustls-pki-types",
@@ -2051,7 +2043,7 @@ dependencies = [
"libc",
"libspa-sys",
"nom 8.0.0",
"rustix 1.1.4",
"rustix 1.1.5",
"system-deps",
]
@@ -2130,9 +2122,9 @@ dependencies = [
[[package]]
name = "lru"
version = "0.18.4"
version = "0.18.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff9840bcc50b71349309900da0ce7279aa336ae71d73250b07998932c7d97c25"
checksum = "ef9ac18847474e638e3702b76c65d4eb93428471a74778ef0f1be711717f89b5"
dependencies = [
"hashbrown 0.17.1",
]
@@ -2227,7 +2219,7 @@ dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -2249,7 +2241,7 @@ dependencies = [
"n0-error",
"n0-future",
"ndk-context",
"rand 0.10.2",
"rand 0.10.3",
"reqwest",
"rustc-hash",
"rustls",
@@ -2356,7 +2348,6 @@ dependencies = [
"wayland-backend",
"wayland-client",
"wayland-protocols",
"wayland-scanner",
]
[[package]]
@@ -2660,7 +2651,7 @@ dependencies = [
"getrandom 0.4.3",
"identity-hash",
"lru-slab",
"rand 0.10.2",
"rand 0.10.3",
"rand_pcg",
"ring",
"rustc-hash",
@@ -2963,7 +2954,7 @@ dependencies = [
"libspa",
"libspa-sys",
"pipewire-sys",
"rustix 1.1.4",
"rustix 1.1.5",
]
[[package]]
@@ -2980,7 +2971,7 @@ dependencies = [
[[package]]
name = "pixelforge"
version = "0.9.1"
source = "git+https://github.com/DatCaptainHorse/pixelforge.git?rev=681fa4dd8bce5dabf008d00983e991b0eb8b3696#681fa4dd8bce5dabf008d00983e991b0eb8b3696"
source = "git+https://github.com/DatCaptainHorse/pixelforge.git?rev=b4b7b36af6f9cbd0fedae220915edd75de933743#b4b7b36af6f9cbd0fedae220915edd75de933743"
dependencies = [
"ash",
"futures-channel",
@@ -3028,7 +3019,7 @@ dependencies = [
"concurrent-queue",
"hermit-abi 0.5.3",
"pin-project-lite",
"rustix 1.1.4",
"rustix 1.1.5",
"windows-sys 0.61.2",
]
@@ -3085,7 +3076,7 @@ dependencies = [
"n0-future",
"netwatch",
"num_enum",
"rand 0.10.2",
"rand 0.10.3",
"serde",
"smallvec",
"socket2",
@@ -3233,9 +3224,9 @@ dependencies = [
[[package]]
name = "rand"
version = "0.10.2"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80"
checksum = "65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af"
dependencies = [
"chacha20",
"getrandom 0.4.3",
@@ -3395,9 +3386,9 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.1.4"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
dependencies = [
"bitflags 2.13.2",
"errno",
@@ -3601,7 +3592,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -3779,7 +3770,7 @@ dependencies = [
"libc",
"profiling",
"rand 0.9.5",
"rustix 1.1.4",
"rustix 1.1.5",
"scopeguard",
"sha2 0.10.9",
"smallvec",
@@ -3889,9 +3880,9 @@ dependencies = [
[[package]]
name = "syn"
version = "3.0.5"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9"
checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
dependencies = [
"proc-macro2",
"quote",
@@ -3915,7 +3906,7 @@ checksum = "901704edd0dfe137f1987838ee4f259e4e063c31371bdb423f7ae38ec6f77f02"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -3967,7 +3958,7 @@ dependencies = [
"fastrand",
"getrandom 0.4.3",
"once_cell",
"rustix 1.1.4",
"rustix 1.1.5",
"windows-sys 0.61.2",
]
@@ -4008,7 +3999,7 @@ checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -4092,7 +4083,7 @@ checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
@@ -4158,7 +4149,7 @@ dependencies = [
"getrandom 0.4.3",
"http",
"httparse",
"rand 0.10.2",
"rand 0.10.3",
"ring",
"rustls-pki-types",
"sha1_smol",
@@ -4393,9 +4384,9 @@ dependencies = [
[[package]]
name = "unicode-ident"
version = "1.0.24"
version = "1.0.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
[[package]]
name = "unicode-segmentation"
@@ -4450,9 +4441,9 @@ dependencies = [
[[package]]
name = "ureq-proto"
version = "0.6.3"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b0809a01d1ca5a51ca70db32bb2a19157582a526505ef3c19e3b343a59aa5ad"
checksum = "f86fd172ccca569e458f61b6bdd6220965a9ef36e672a6852953b51a0e1583be"
dependencies = [
"base64 0.23.1",
"http",
@@ -4606,7 +4597,7 @@ dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
"wasm-bindgen-shared",
]
@@ -4640,7 +4631,7 @@ checksum = "38a91b4eaddff87b1cd1074985e3713da4af2c49742d1b356b2c01670a67a078"
dependencies = [
"cc",
"downcast-rs",
"rustix 1.1.4",
"rustix 1.1.5",
"smallvec",
"wayland-sys",
]
@@ -4652,7 +4643,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3c36a0f861ad76d0901f2800b46321410d9f73f2ea88aac0650d86c32688073"
dependencies = [
"bitflags 2.13.2",
"rustix 1.1.4",
"rustix 1.1.5",
"wayland-backend",
"wayland-scanner",
]
@@ -4715,7 +4706,7 @@ checksum = "0dde9c29be0f723a573977de51ee455bf3dfa03652730a74f9dd3b337e374d75"
dependencies = [
"bitflags 2.13.2",
"downcast-rs",
"rustix 1.1.4",
"rustix 1.1.5",
"wayland-backend",
"wayland-scanner",
]
@@ -5219,7 +5210,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9993aa5be5a26815fe2c3eacfc1fde061fc1a1f094bf1ad2a18bf9c495dd7414"
dependencies = [
"gethostname",
"rustix 1.1.4",
"rustix 1.1.5",
"x11rb-protocol",
]
@@ -5280,7 +5271,7 @@ checksum = "33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
"synstructure",
]
@@ -5321,7 +5312,7 @@ checksum = "f75b4683f6c7f45248d4d64056a24298c6281e0993356d7d1b4a1a962ef10d4a"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
"synstructure",
]
@@ -5375,7 +5366,7 @@ checksum = "34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.5",
"syn 3.0.6",
]
[[package]]
apps/nescapture/Cargo.toml
+2 -2
View File
@@ -32,9 +32,9 @@ toml = "0.8"
serde = { version = "1", features = ["derive"] }
# Vulkan Video hardware encoding.
pixelforge = { git = "https://github.com/DatCaptainHorse/pixelforge.git", rev = "681fa4dd8bce5dabf008d00983e991b0eb8b3696", features = ["dmabuf"] }
pixelforge = { git = "https://github.com/DatCaptainHorse/pixelforge.git", rev = "b4b7b36af6f9cbd0fedae220915edd75de933743" }
# libc for DMA-BUF OS primitives
# libc for socket options
libc = "0.2"
# Shared IPC protocol
apps/nescapture/README.md
+84 -13
View File
@@ -24,15 +24,12 @@ Game process
│ vkCmdBindPipeline → detect HUD │
│ vkQueuePresentKHR → capture+encode │
└──────────────────────────────────────────────┘
│ GPU blit, same device
│ GPU blit on the game's queue, signals a timeline semaphore
▼
final_image (DMA-BUF exportable)
│ get_dmabuf_fd(final_memory)
ring slot (an image on the game's own device)
│ read in place, once the blit's point is reached
▼
DmaBufImporter (pixelforge VkDevice)
│ import_or_reuse() → vk::Image
▼
ColorConverter (GPU compute shader)
ColorConverter (GPU compute shader, the encoder's own queue)
│ BGRA/RGB10/FP16 → NV12/P010/YUV444
▼
Encoder (Vulkan Video: H.264 / H.265 / AV1)
@@ -41,8 +38,24 @@ Encoder (Vulkan Video: H.264 / H.265 / AV1)
Unix datagram → neshub → the client
```
CPU fallback exists only for driver configurations without DMA-BUF external
memory export.
The encoder runs on the game's own `VkDevice`. The layer creates that device
with what the encoder needs: the extensions and feature bits it asks for, and
queues of its own wherever a queue family has one to spare, since a `VkQueue`
may not be submitted to from two threads at once. Where no family has room,
the game's queue is created internally synchronized and shared. Every step of
a frame is ordered on the GPU; nothing waits on the CPU.
Where the conversion is only the YUV matrix and the device has
`VK_VALVE_video_encode_rgb_conversion`, the encoder takes the RGB frame and
converts it itself, and the converter is not built. That path runs at limited
range, since the one driver offering it writes limited range whatever it is
asked; everything else is full range.
When the game's device cannot host the encoder -- an instance the loader
cannot raise to Vulkan 1.1, no queue the encoder could safely use, a driver
refusing the additions, or `NESCAPTURE_SHARED_DEVICE=0` -- the encoder gets a
device of its own, and each frame is read back on the CPU and uploaded there.
That works everywhere and costs a copy each way.
---
@@ -93,11 +106,18 @@ implicit layer is loaded into *every* Vulkan process on the system.
| `NESCAPTURE_CODEC` | best available | `h264`, `h265` or `av1`; probes if unset |
| `NESCAPTURE_FORMAT` | `yuv420` | `yuv420` or `yuv444` |
| `NESCAPTURE_DEPTH` | auto | `8` or `10`; inferred from the swapchain `VkFormat` if unset |
| `NESCAPTURE_BITRATE` | `10000` | CBR target in kbps. Ignored when `NESCAPTURE_QP` is set |
| `NESCAPTURE_QP` | _(unset)_ | Constant QP instead of CBR |
| `NESCAPTURE_RC` | _(inferred)_ | `cqp`, `cbr` or `vbr`. Unset infers `cqp` when `NESCAPTURE_QP` is set, `cbr` otherwise |
| `NESCAPTURE_BITRATE` | `10000` | Target bitrate in kbps, under `cbr` and `vbr` |
| `NESCAPTURE_BITRATE_MAX` | 1.5x the target | VBR ceiling in kbps. Ignored outside `vbr` |
| `NESCAPTURE_QP` | _(unset)_ | Constant QP, under `cqp` |
| `NESCAPTURE_FPS` | `60` | Target frame rate |
| `NESCAPTURE_IDR_INTERVAL` | `4` | Force an IDR every N **seconds** |
| `NESCAPTURE_INTRA_REFRESH` | _(off)_ | Set to `1` to replace periodic key frames with an intra refresh cycle |
| `NESCAPTURE_INTRA_REFRESH_QP_DELTA` | `-4` | QP shift inside the refresh band; negative spends bits on it |
| `NESCAPTURE_INTRA_REFRESH_SHAPE` | auto | `rows`, `columns` or `partitions`; the driver chooses if unset |
| `NESCAPTURE_TUNE` | _(unset)_ | `highquality`, `lowlatency`, `ultralowlatency`, `lossless` |
| `NESCAPTURE_SHARED_DEVICE` | _(on)_ | Set to `0` to leave the game's device as the game asked for it, and encode on a device of the encoder's own with CPU readback |
| `NESCAPTURE_RGB_ENCODE` | _(on)_ | Set to `0` to always convert with the shader, even where the encoder could convert RGB itself |
| `NESCAPTURE_CONFIG` | _(unset)_ | Path to the per-app shader-hash TOML |
| `NESCAPTURE_GAME_NAME` | exe basename | Override app identification for that config |
| `NESCAPTURE_DISCOVER` | _(unset)_ | Set to `1` to log every draw, for finding HUD shaders |
@@ -107,6 +127,57 @@ Everything else is decided at runtime: the client asks `neshub` for a codec or
bitrate change and it arrives on the command socket, so the encoder is
reconfigured without a restart.
### Intra refresh
Instead of a key frame every few seconds, each picture codes one slice of the
image as intra, so after a full cycle every part has been refreshed. The same
cost, paid evenly, with no picture much larger than any other — which is what
a link with a latency budget wants, since a key frame is the largest frame
there is.
**The cycle length is not configurable, deliberately.** It is bounded by how
many refresh regions the picture actually has, and that depends on the codec's
block size: at 1080p an H.265 picture is 17 CTB rows tall where an H.264 one is
68 macroblock rows, so the same duration is comfortable for one codec and
impossible for the other. The encoder knows the codec, the resolution and what
the device allows, and derives it from the key frame interval it replaces.
The refresh here spreads cost only; it does not make the cycle a recovery
point. Doing that would restrict prediction on every picture — expensive, and
what turns the refreshed band into a visible discontinuity — to buy a
guarantee this stream gets more cheaply from the client asking for an IDR.
The band is coded intra every cycle, so it carries none of the refinement its
neighbours have accumulated and reads as a strip of lower quality sweeping
across the picture. `NESCAPTURE_INTRA_REFRESH_QP_DELTA` spends bits back into
it, out of the rest of the frame. Measured at 1080p with ColorVideoVDP, `-4`
recovers a fifth of what intra refresh costs and the encoded size does not
grow — but the best value depends on the content, and too large a shift
starves the rest of the frame faster than too small a one helps. Devices that
cannot express a negative delta, or whose refresh regions follow the slice
layout rather than a block sweep, decline it and say so.
`NESCAPTURE_INTRA_REFRESH_SHAPE` stays configurable because the device cannot
answer it: whether a horizontal or vertical sweep looks better depends on how
the content moves. It also changes how many regions there are — a 1080p
picture in 64×64 blocks is 17 rows but 30 columns, so `columns` allows a
longer cycle and thus less intra per picture.
### Rate control
`cbr` holds every frame to the same size, which is what a link with a fixed
budget wants. `vbr` holds the same *average* while letting a frame that needs
it spend up to the ceiling — a scene change is coded rather than smeared, at
the cost of a burst the path has to absorb. `cqp` holds quality constant and
lets the bitrate go wherever the content takes it, which is a recording
setting rather than a streaming one.
The command socket carries a target bitrate but has no way to name a mode
beyond CBR and constant QP. A target arriving while the encode is `vbr` is
therefore applied as a target, leaving the mode and the ceiling alone — so the
ceiling asked for at launch survives a session, and a congestion controller
adjusts underneath it. Retargeting costs no rebuild and no key frame.
---
## Per-app shader-hash config
@@ -145,10 +216,10 @@ src/
├── framebuffer.rs image view and framebuffer tracking
├── commands.rs vkCmdBind*, vkCmdDraw*, vkCmdBeginRenderPass
├── swapchain.rs vkCreateSwapchainKHR, image enumeration
├── capture.rs GPU blit to the capture image, DMA-BUF export
├── capture.rs GPU blit into the capture ring, CPU readback fallback
├── present.rs vkQueuePresentKHR, encode dispatch
├── encode.rs pixelforge pipeline, codec probing, IPC send
├── dmabuf_import.rs cross-device zero-copy import
├── shared.rs creating the game's device for the encoder to share
├── config.rs per-app TOML shader-hash config
└── discovery.rs draw-call logging for shader discovery
```
apps/nescapture/scripts/verify-chain.sh
+3 -2
View File
@@ -4,7 +4,7 @@
# Runs a Vulkan workload under the compositor with the layer active, then checks
# the encoded result against the compositor's own readback of the same frames.
# Two independent paths see the same content: the compositor reads the surface
# back to the CPU, the layer exports it as a DMA-BUF and encodes it on the GPU.
# back to the CPU, the layer encodes it on the GPU without it leaving the device.
# Agreement between them is the evidence; a single path cannot tell a correct
# frame from a plausible-looking wrong one.
#
@@ -96,7 +96,8 @@ wait $RECV || true
FRAMES="$(cat "$WORK/frames.txt")"
echo
echo "frames encoded: $FRAMES"
grep -m1 "First import" "$WORK/run.log" || echo " (no DMA-BUF import logged)"
grep -m1 "encoding on the game's own device" "$WORK/run.log" \
|| echo " (encoding on a device of its own, with CPU readback)"
python3 - "$WORK" "$STREAM" "$FRAMES" <<'PY'
import glob, subprocess, sys
apps/nescapture/src/capture.rs
+818 -287
View File
File diff suppressed because it is too large Load Diff
apps/nescapture/src/device.rs
+163 -86
View File
@@ -14,6 +14,7 @@ use std::os::raw::c_void;
use std::sync::Arc;
const VK_LAYER_LINK_INFO: u32 = 0;
const VK_LOADER_DATA_CALLBACK: u32 = 1;
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkCreateDevice(
@@ -43,6 +44,14 @@ pub unsafe extern "system" fn vkCreateDevice(
};
unsafe { (*layer_info).u.pDeviceLayerInfo = (*dev_link).pNext };
let set_loader_data = unsafe {
find_layer_link::<VkLayerDeviceCreateInfo>(
(*p_create_info).p_next as *const c_void,
VK_LOADER_DATA_CALLBACK,
)
}
.and_then(|info| unsafe { (*info).u.pfnSetDeviceLoaderData });
let inst_key = unsafe { dispatch_key(physical_device.as_raw() as *const c_void) };
let istate = match INSTANCE_STATE.get(&inst_key) {
Some(s) => s.clone(),
@@ -54,11 +63,8 @@ pub unsafe extern "system" fn vkCreateDevice(
}
};
// ── Inject DMA-BUF extensions for zero-copy capture ──────────────────
let ci = unsafe { &*p_create_info };
// Collect the game's original extensions.
let original_extensions: Vec<*const libc::c_char> =
let game_extensions: Vec<*const libc::c_char> =
if ci.enabled_extension_count > 0 && !ci.pp_enabled_extension_names.is_null() {
unsafe {
std::slice::from_raw_parts(
@@ -71,69 +77,34 @@ pub unsafe extern "system" fn vkCreateDevice(
Vec::new()
};
// Extensions we need — static byte strings so pointers stay valid.
const EXT_EXTERNAL_MEMORY: &[u8] = b"VK_KHR_external_memory\0";
const EXT_EXTERNAL_MEMORY_FD: &[u8] = b"VK_KHR_external_memory_fd\0";
const EXT_EXTERNAL_MEMORY_DMABUF: &[u8] = b"VK_EXT_external_memory_dma_buf\0";
// Lets the capture ring be allocated tiled. The importer has always created
// its side with DRM_FORMAT_MODIFIER_EXT tiling; without this the producer
// can only offer it a linear buffer.
const EXT_IMAGE_DRM_FORMAT_MODIFIER: &[u8] = b"VK_EXT_image_drm_format_modifier\0";
let needed: &[&[u8]] = &[
EXT_EXTERNAL_MEMORY,
EXT_EXTERNAL_MEMORY_FD,
EXT_EXTERNAL_MEMORY_DMABUF,
EXT_IMAGE_DRM_FORMAT_MODIFIER,
];
// Build extended list: original + any of ours not already present.
let mut extended = original_extensions.clone();
for &ext in needed {
let name_cstr = unsafe { std::ffi::CStr::from_bytes_with_nul_unchecked(ext) };
let already = extended
.iter()
.any(|&ptr| unsafe { std::ffi::CStr::from_ptr(ptr) == name_cstr });
if !already {
extended.push(ext.as_ptr() as *const libc::c_char);
// Where it can, the encoder runs on this very device, so the device is
// first created with what that needs. Anything refused falls back to the
// device exactly as the game asked for it, and the encoder to a device of
// its own.
let mut shared = None;
if let Some(prepared) =
unsafe { crate::shared::prepare(&istate, next_gdpa, physical_device, ci, &game_extensions) }
{
let shared_ci = prepared.create_info(ci);
let result =
unsafe { (istate.create_device)(physical_device, &shared_ci, p_allocator, p_device) };
let (additions, entry, instance) = unsafe { prepared.finish() };
if result == vk::Result::SUCCESS {
shared = Some((additions, entry, instance));
} else {
log::warn!(
"vkCreateDevice with the encoder's additions failed ({result:?}), \
retrying as the game asked"
);
}
}
// Try with injected extensions first.
//
// The device's queue create info is passed through unchanged. An earlier
// version bumped the first family's queue count by one to get a dedicated
// capture queue, which was then never used — and could not be: the capture
// blit has to be submitted to the queue the game presents on, or it gains
// no ordering against the present. All the bump did was risk exceeding the
// family's available queue count on the way in.
let mut modified_ci = *ci;
modified_ci.enabled_extension_count = extended.len() as u32;
modified_ci.pp_enabled_extension_names = extended.as_ptr();
let mut dmabuf_available = true;
let result =
unsafe { (istate.create_device)(physical_device, &modified_ci, p_allocator, p_device) };
let result = if result != vk::Result::SUCCESS {
// Driver rejected our extensions — retry with original create info.
log::warn!(
"vkCreateDevice with DMA-BUF extensions failed ({:?}), \
retrying without — CPU readback fallback will be used",
result
);
dmabuf_available = false;
unsafe { (istate.create_device)(physical_device, p_create_info, p_allocator, p_device) }
} else {
log::info!("DMA-BUF extensions injected successfully");
result
};
if result != vk::Result::SUCCESS {
return result;
}
if !dmabuf_available {
log::warn!("DMA-BUF extensions missing — will use CPU readback fallback (expensive!)");
if shared.is_none() {
let result = unsafe {
(istate.create_device)(physical_device, p_create_info, p_allocator, p_device)
};
if result != vk::Result::SUCCESS {
return result;
}
}
let device = unsafe { *p_device };
@@ -156,6 +127,7 @@ pub unsafe extern "system" fn vkCreateDevice(
get_device_proc_addr: next_gdpa,
destroy_device: load!(b"vkDestroyDevice\0"),
get_device_queue: load!(b"vkGetDeviceQueue\0"),
get_device_queue2: try_load!(b"vkGetDeviceQueue2\0"),
queue_present_khr: try_load!(b"vkQueuePresentKHR\0"),
// Phase 1
@@ -189,10 +161,6 @@ pub unsafe extern "system" fn vkCreateDevice(
cmd_pipeline_barrier: load!(b"vkCmdPipelineBarrier\0"),
cmd_copy_image: load!(b"vkCmdCopyImage\0"),
get_image_subresource_layout: load!(b"vkGetImageSubresourceLayout\0"),
get_memory_fd_khr: try_load!(b"vkGetMemoryFdKHR\0"),
get_image_drm_format_modifier_properties_ext: try_load!(
b"vkGetImageDrmFormatModifierPropertiesEXT\0"
),
create_query_pool: try_load!(b"vkCreateQueryPool\0"),
destroy_query_pool: try_load!(b"vkDestroyQueryPool\0"),
cmd_reset_query_pool: try_load!(b"vkCmdResetQueryPool\0"),
@@ -232,6 +200,17 @@ pub unsafe extern "system" fn vkCreateDevice(
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
let shared = shared.map(|(additions, entry, instance)| unsafe {
crate::shared::SharedDevice::adopt(
additions,
entry,
instance,
physical_device,
device,
next_gdpa,
)
});
// Phase 3: load shader hash config
let shader_hashes = config::resolve_config_path()
.as_ref()
@@ -252,6 +231,9 @@ pub unsafe extern "system" fn vkCreateDevice(
raw: device,
physical_device,
fp,
shared,
shared_active: std::sync::atomic::AtomicBool::new(false),
set_loader_data,
shader_registry: DashMap::new(),
pipeline_registry: DashMap::new(),
@@ -279,6 +261,7 @@ pub unsafe extern "system" fn vkCreateDevice(
}),
swapchain_colorspace: std::sync::atomic::AtomicU32::new(0),
frame_counter: std::sync::atomic::AtomicU64::new(0),
ring_generation: std::sync::atomic::AtomicU64::new(0),
hud_detected_frame: std::sync::atomic::AtomicBool::new(false),
pending_capture_frame: std::sync::atomic::AtomicBool::new(false),
@@ -287,7 +270,6 @@ pub unsafe extern "system" fn vkCreateDevice(
encoder: std::sync::Mutex::new(None),
frame_gate: std::sync::Mutex::new(crate::pacing::FrameGate::from_env()),
frame_pacer: std::sync::Mutex::new(crate::pacing::FramePacer::from_env()),
last_present_return: std::sync::Mutex::new(None),
@@ -317,23 +299,20 @@ pub unsafe extern "system" fn vkDestroyDevice(
ds.pipeline_registry.len(),
);
// ── 1. Shut down encoder pipeline (unblocks encoder + RTP threads) ───
{
let mut enc_guard = ds.encoder.lock().unwrap();
if let Some(handle) = enc_guard.take() {
handle.shutdown();
// `handle` is dropped here → drops `frame_tx` → encoder thread's
// recv_timeout returns Disconnected → encoder thread drops
// `encoded_tx` → RTP thread exits too.
//
// Give threads a moment to drain. In production you'd join the
// JoinHandles, but since we don't store them, a short sleep +
// the AtomicBool shutdown flag is sufficient.
log::info!("encoder pipeline shutdown signaled");
// ── 1. Shut down the encode pipeline ──────────────────────────────────
//
// Waited for, not just signalled. On a shared device the encoder, its
// converter and their images are objects on this very device, and
// destroying the device under them is a use-after-free. The receive
// timeout bounds how long the thread takes to notice.
let handle = ds.encoder.lock().ok().and_then(|mut g| g.take());
if let Some(handle) = handle {
if handle.finish(std::time::Duration::from_secs(2)) {
log::info!("encoder pipeline stopped");
} else {
log::error!("encoder thread did not stop in time; destroying the device anyway");
}
}
// Brief yield to let threads notice the disconnect.
std::thread::sleep(std::time::Duration::from_millis(50));
// ── 2. Tear down the capture ring ─────────────────────────────────────
//
@@ -392,13 +371,111 @@ pub unsafe extern "system" fn vkGetDeviceQueue(
) {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
if let Some(ds) = DEVICE_STATE.get(&key) {
unsafe { (ds.fp.get_device_queue)(device, queue_family_index, queue_index, p_queue) };
// A queue created internally synchronized, so the encoder can share
// it, is invisible to vkGetDeviceQueue: only vkGetDeviceQueue2 with the
// matching flags returns it. The game asked for a plain queue and gets
// this one.
match (
shares_family(&ds, queue_family_index),
ds.fp.get_device_queue2,
) {
(true, Some(get2)) => {
let info = vk::DeviceQueueInfo2::default()
.flags(vk::DeviceQueueCreateFlags::INTERNALLY_SYNCHRONIZED_KHR)
.queue_family_index(queue_family_index)
.queue_index(queue_index);
unsafe { get2(device, &info, p_queue) };
}
_ => unsafe {
(ds.fp.get_device_queue)(device, queue_family_index, queue_index, p_queue)
},
}
let queue = unsafe { *p_queue };
// The encoder's queues reach this hook without the loader in between.
// For the game's, the loader stamps them again on the way out.
unsafe { stamp(&ds, queue.as_raw() as *mut c_void) };
QUEUE_TO_DEVICE_KEY.insert(queue.as_raw(), key);
crate::state::QUEUE_TO_FAMILY.insert(queue.as_raw(), queue_family_index);
}
}
/// Give a dispatchable object this layer obtained itself the loader's
/// dispatch data.
///
/// The loader writes its dispatch pointer into every queue and command buffer
/// that passes through its own entry points. One a layer allocates by calling
/// the next layer directly never does, and a layer below this one that finds
/// its per-object state by that pointer then finds nothing: the validation
/// layer aborts, in the first vkCmd* recorded into such a command buffer. The
/// loader hands every layer this callback at vkCreateDevice for exactly this.
///
/// # Safety
///
/// `object` must be a queue or command buffer of `ds`'s device.
pub unsafe fn stamp(ds: &DeviceState, object: *mut c_void) {
if let Some(set) = ds.set_loader_data
&& unsafe { set(ds.raw, object) } != vk::Result::SUCCESS
{
log::warn!("vkSetDeviceLoaderData refused an object of the layer's own");
}
}
/// vkAllocateCommandBuffers for the encoder, whose command buffers are the
/// layer's own: allocated below the loader, so stamped here. See [`stamp`].
pub unsafe extern "system" fn encoder_allocate_command_buffers(
device: vk::Device,
p_allocate_info: *const vk::CommandBufferAllocateInfo<'_>,
p_command_buffers: *mut vk::CommandBuffer,
) -> vk::Result {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
let Some(ds) = DEVICE_STATE.get(&key).map(|s| s.clone()) else {
return vk::Result::ERROR_INITIALIZATION_FAILED;
};
let result =
unsafe { (ds.fp.allocate_command_buffers)(device, p_allocate_info, p_command_buffers) };
if result == vk::Result::SUCCESS {
let count = unsafe { (*p_allocate_info).command_buffer_count } as usize;
for cb in unsafe { std::slice::from_raw_parts(p_command_buffers, count) } {
unsafe { stamp(&ds, cb.as_raw() as *mut c_void) };
}
}
result
}
/// Whether the game's queues in `family` were created internally synchronized
/// for the encoder to share.
fn shares_family(ds: &DeviceState, family: u32) -> bool {
ds.shared
.as_ref()
.is_some_and(|s| s.queues.internally_synchronized.contains(&family))
}
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkGetDeviceQueue2(
device: vk::Device,
p_queue_info: *const vk::DeviceQueueInfo2,
p_queue: *mut vk::Queue,
) {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
let Some(ds) = DEVICE_STATE.get(&key) else {
return;
};
let Some(get2) = ds.fp.get_device_queue2 else {
return;
};
// The flags have to match the ones the queue was created with, and for a
// shared family the layer added one the game does not know about.
let mut info = unsafe { *p_queue_info };
if shares_family(&ds, info.queue_family_index) {
info.flags |= vk::DeviceQueueCreateFlags::INTERNALLY_SYNCHRONIZED_KHR;
}
unsafe { get2(device, &info, p_queue) };
let queue = unsafe { *p_queue };
unsafe { stamp(&ds, queue.as_raw() as *mut c_void) };
QUEUE_TO_DEVICE_KEY.insert(queue.as_raw(), key);
crate::state::QUEUE_TO_FAMILY.insert(queue.as_raw(), info.queue_family_index);
}
/// Enumerate device extensions supported by the physical device.
unsafe fn enumerate_device_extensions(
istate: &crate::dispatch::NextInstanceFn,
apps/nescapture/src/discovery.rs
+2 -2
View File
@@ -2,7 +2,7 @@
// discovery.rs — Phase 6: per-draw logging for shader hash discovery
//
// When HUDLESS_DISCOVER=1 is set, every vkCmdDraw* call is logged to:
// /tmp/hudless_discover_$EXE.log
// /tmp/nescapture_discover_$EXE.log
//
// Log format:
// frame=0001 draw=00042 vert=0x1a2b3c4d5e6f7890 frag=0xaabbccddeeff0011 verts=6 blend=true depth=false
@@ -40,7 +40,7 @@ fn ensure_log_file() -> bool {
.unwrap_or_else(|| "unknown".to_string())
});
let path = format!("/tmp/hudless_discover_{}.log", exe_name);
let path = format!("/tmp/nescapture_discover_{}.log", exe_name);
match OpenOptions::new().create(true).append(true).open(&path) {
Ok(f) => {
log::info!("discovery logging to {}", path);
apps/nescapture/src/dispatch.rs
+14 -45
View File
@@ -38,6 +38,8 @@ pub type PFN_vkDestroyDevice =
unsafe extern "system" fn(vk::Device, *const vk::AllocationCallbacks);
pub type PFN_vkGetDeviceQueue = unsafe extern "system" fn(vk::Device, u32, u32, *mut vk::Queue);
pub type PFN_vkGetDeviceQueue2 =
unsafe extern "system" fn(vk::Device, *const vk::DeviceQueueInfo2, *mut vk::Queue);
pub type PFN_vkQueuePresentKHR =
unsafe extern "system" fn(vk::Queue, *const vk::PresentInfoKHR) -> vk::Result;
@@ -190,20 +192,6 @@ pub type PFN_vkGetImageSubresourceLayout = unsafe extern "system" fn(
*mut vk::SubresourceLayout,
);
// DRM format modifiers. Both optional: without them the capture ring stays
// linear, which is what it was before it could be anything else.
pub type PFN_vkGetPhysicalDeviceFormatProperties2 = unsafe extern "system" fn(
vk::PhysicalDevice,
vk::Format,
*mut vk::FormatProperties2<'_>,
);
pub type PFN_vkGetImageDrmFormatModifierPropertiesEXT = unsafe extern "system" fn(
vk::Device,
vk::Image,
*mut vk::ImageDrmFormatModifierPropertiesEXT<'_>,
) -> vk::Result;
// Timestamp queries around the capture blit. All optional: without them the
// blit's GPU cost is simply not reported.
pub type PFN_vkGetPhysicalDeviceProperties =
@@ -219,21 +207,14 @@ pub type PFN_vkCreateQueryPool = unsafe extern "system" fn(
*mut vk::QueryPool,
) -> vk::Result;
pub type PFN_vkDestroyQueryPool = unsafe extern "system" fn(
vk::Device,
vk::QueryPool,
*const vk::AllocationCallbacks,
);
pub type PFN_vkDestroyQueryPool =
unsafe extern "system" fn(vk::Device, vk::QueryPool, *const vk::AllocationCallbacks);
pub type PFN_vkCmdResetQueryPool =
unsafe extern "system" fn(vk::CommandBuffer, vk::QueryPool, u32, u32);
pub type PFN_vkCmdWriteTimestamp = unsafe extern "system" fn(
vk::CommandBuffer,
vk::PipelineStageFlags,
vk::QueryPool,
u32,
);
pub type PFN_vkCmdWriteTimestamp =
unsafe extern "system" fn(vk::CommandBuffer, vk::PipelineStageFlags, vk::QueryPool, u32);
pub type PFN_vkGetQueryPoolResults = unsafe extern "system" fn(
vk::Device,
@@ -246,13 +227,6 @@ pub type PFN_vkGetQueryPoolResults = unsafe extern "system" fn(
vk::QueryResultFlags,
) -> vk::Result;
// DMA-BUF fd export (used to share final_image with pixelforge zero-copy)
pub type PFN_vkGetMemoryFdKHR = unsafe extern "system" fn(
vk::Device,
*const vk::MemoryGetFdInfoKHR,
*mut std::os::raw::c_int,
) -> vk::Result;
// ── Phase 4: Synchronisation ─────────────────────────────────────────────────
pub type PFN_vkCreateFence = unsafe extern "system" fn(
@@ -366,13 +340,14 @@ pub type PFN_vkResetCommandBuffer =
// ── Dispatch table structs ────────────────────────────────────────────────────
pub struct NextInstanceFn {
/// The instance itself, for building the encoder's view of it.
pub instance: vk::Instance,
/// The Vulkan version the application asked for, which bounds what core
/// functionality exists on its devices.
pub api_version: u32,
pub get_instance_proc_addr: PFN_vkGetInstanceProcAddr,
pub destroy_instance: PFN_vkDestroyInstance,
pub get_physical_device_memory_properties: PFN_vkGetPhysicalDeviceMemoryProperties,
/// `None` on an instance below Vulkan 1.1 without
/// `VK_KHR_get_physical_device_properties2`. Without it the modifier list
/// cannot be queried and the capture ring stays linear.
pub get_physical_device_format_properties2: Option<PFN_vkGetPhysicalDeviceFormatProperties2>,
/// Needed for `timestampPeriod`, which turns device ticks into nanoseconds.
pub get_physical_device_properties: Option<PFN_vkGetPhysicalDeviceProperties>,
/// Needed for a queue family's `timestampValidBits`. A family reporting
@@ -388,6 +363,9 @@ pub struct NextDeviceFn {
pub get_device_proc_addr: PFN_vkGetDeviceProcAddr,
pub destroy_device: PFN_vkDestroyDevice,
pub get_device_queue: PFN_vkGetDeviceQueue,
/// Core in 1.1. Needed to fetch queues created with flags, which is how a
/// queue shared with the encoder is created.
pub get_device_queue2: Option<PFN_vkGetDeviceQueue2>,
pub queue_present_khr: Option<PFN_vkQueuePresentKHR>,
// Phase 1
@@ -421,15 +399,6 @@ pub struct NextDeviceFn {
pub cmd_pipeline_barrier: PFN_vkCmdPipelineBarrier,
pub cmd_copy_image: PFN_vkCmdCopyImage,
pub get_image_subresource_layout: PFN_vkGetImageSubresourceLayout,
/// `None` when `VK_KHR_external_memory_fd` is unavailable.
/// Required for DMA-BUF export to pixelforge's VkDevice.
pub get_memory_fd_khr: Option<PFN_vkGetMemoryFdKHR>,
/// `None` when `VK_EXT_image_drm_format_modifier` was not enabled. The
/// driver picks the modifier from the list it is offered, so this is how
/// the layer learns which one it actually got — and the importer needs the
/// exact value, not the list.
pub get_image_drm_format_modifier_properties_ext:
Option<PFN_vkGetImageDrmFormatModifierPropertiesEXT>,
// Phase 4 — blit timing. All-or-nothing: the ring only times the blit when
// every one of these loaded and the presenting queue family can timestamp.
apps/nescapture/src/dmabuf_import.rs
-259
View File
@@ -1,259 +0,0 @@
//! DMA-BUF import support for zero-copy video encoding.
//!
//! This module provides the ability to import Linux DMA-BUF file descriptors as
//! Vulkan images for direct video encoding without CPU-side copies.
//!
//! `DmaBufImporter` caches imported Vulkan resources per compositor buffer index
//! so that pre-allocated GBM buffers are imported only once. Subsequent frames
//! from the same buffer reuse the cached `VkImage` and `VkDeviceMemory`,
//! eliminating per-frame Vulkan object creation and layout transitions.
use anyhow::Result;
use ash::vk;
use log::debug;
use pixelforge::VideoContext;
use std::os::fd::RawFd;
use std::os::unix::io::{BorrowedFd, IntoRawFd};
/// Information about a single DMA-BUF plane.
#[derive(Debug, Clone, Copy)]
pub struct DmaBufPlane {
/// File descriptor for the DMA-BUF.
pub fd: RawFd,
/// Offset within the DMA-BUF to the start of this plane.
pub offset: u32,
/// Row stride in bytes.
pub stride: u32,
/// DRM format modifier.
pub modifier: u64,
}
/// Cached Vulkan resources for a single compositor buffer slot.
struct CachedImport {
image: vk::Image,
memory: vk::DeviceMemory,
}
/// Importer for DMA-BUF file descriptors into Vulkan images.
///
/// Owns a per-buffer-index cache of `VkImage` + `VkDeviceMemory`.
/// Layout transitions are deferred to the consumer (e.g. `ColorConverter`)
/// to avoid a separate GPU submission per first-time import.
pub struct DmaBufImporter {
context: VideoContext,
external_memory_fd: ash::khr::external_memory_fd::Device,
/// Per-buffer-index cache. Index corresponds to `ExportedFrame::buffer_index`.
cached_imports: Vec<Option<CachedImport>>,
}
impl DmaBufImporter {
/// Create a new DMA-BUF importer.
pub fn new(context: VideoContext) -> Result<Self> {
let external_memory_fd =
ash::khr::external_memory_fd::Device::load(context.instance(), context.device());
Ok(Self {
context,
external_memory_fd,
cached_imports: Vec::new(),
})
}
/// Import a DMA-BUF as a Vulkan image, reusing a cached import when
/// the same `buffer_index` has been seen before.
///
/// The `format` parameter specifies the Vulkan format matching the DMA-BUF
/// pixel format (e.g. `B8G8R8A8_UNORM` for SDR, `A2B10G10R10_UNORM_PACK32`
/// for 10-bit HDR, `R16G16B16A16_SFLOAT` for FP16 HDR).
///
/// Returns `(image, needs_transition)` where `needs_transition` is `true`
/// for first-time imports whose image is still in `UNDEFINED` layout.
/// The caller is responsible for transitioning the image (e.g. by passing
/// the appropriate `src_layout` to `ColorConverter::convert`).
pub fn import_or_reuse(
&mut self,
buffer_index: usize,
width: u32,
height: u32,
format: vk::Format,
planes: &[DmaBufPlane],
) -> Result<(vk::Image, bool)> {
// Grow the cache vector if needed.
if self.cached_imports.len() <= buffer_index {
self.cached_imports.resize_with(buffer_index + 1, || None);
}
if let Some(cached) = &self.cached_imports[buffer_index] {
return Ok((cached.image, false));
}
// First time seeing this buffer — full import.
debug!(
"First import for buffer {buffer_index}: {}x{}, format={:?}, fd={}, stride={}, modifier={:#x}",
width, height, format, planes[0].fd, planes[0].stride, planes[0].modifier
);
let (image, memory) = self.import_internal(width, height, format, planes)?;
self.cached_imports[buffer_index] = Some(CachedImport { image, memory });
Ok((image, true))
}
/// Perform the raw Vulkan import of a DMA-BUF with the specified format.
///
/// Returns the `(VkImage, VkDeviceMemory)` pair. The image is in
/// `UNDEFINED` layout; the caller must transition it.
fn import_internal(
&self,
width: u32,
height: u32,
format: vk::Format,
planes: &[DmaBufPlane],
) -> Result<(vk::Image, vk::DeviceMemory)> {
if planes.is_empty() {
return Err(anyhow::anyhow!("At least one DMA-BUF plane is required"));
}
let device = self.context.device();
// Build DRM format modifier plane layouts for all planes.
// AMD modifiers (e.g. tiled/DCC) may require multiple planes;
// the layout count must match the modifier's expected plane count.
let plane_layouts: Vec<vk::SubresourceLayout> = planes
.iter()
.map(|p| {
vk::SubresourceLayout::default()
.offset(p.offset as u64)
.row_pitch(p.stride as u64)
})
.collect();
let modifier = planes[0].modifier;
let mut drm_format_modifier_info =
vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(modifier)
.plane_layouts(&plane_layouts);
let mut external_memory_info = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
external_memory_info.p_next = &mut drm_format_modifier_info
as *mut vk::ImageDrmFormatModifierExplicitCreateInfoEXT
as *mut _;
let mut image_create_info = vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(format)
.extent(vk::Extent3D {
width,
height,
depth: 1,
})
.mip_levels(1)
.array_layers(1)
.samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT)
.usage(vk::ImageUsageFlags::TRANSFER_SRC | vk::ImageUsageFlags::SAMPLED)
.sharing_mode(vk::SharingMode::EXCLUSIVE)
.initial_layout(vk::ImageLayout::UNDEFINED);
image_create_info.p_next =
&mut external_memory_info as *mut vk::ExternalMemoryImageCreateInfo as *mut _;
let image = unsafe { device.create_image(&image_create_info, None) }
.map_err(|e| anyhow::anyhow!("DMA-BUF image creation: {e}"))?;
// Memory requirements.
let mem_requirements = unsafe { device.get_image_memory_requirements(image) };
// FD memory properties.
let mut memory_fd_properties = vk::MemoryFdPropertiesKHR::default();
unsafe {
self.external_memory_fd.get_memory_fd_properties(
vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT,
planes[0].fd,
&mut memory_fd_properties,
)
}
.map_err(|e| anyhow::anyhow!("Failed to get memory FD properties: {e}"))?;
// Duplicate the FD — vkAllocateMemory consumes it.
let fd = unsafe { BorrowedFd::borrow_raw(planes[0].fd) }
.try_clone_to_owned()
.map_err(|e| anyhow::anyhow!("Failed to duplicate DMA-BUF FD: {e}"))?
.into_raw_fd();
let mut import_memory_fd_info = vk::ImportMemoryFdInfoKHR::default()
.handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT)
.fd(fd);
let memory_type_bits =
mem_requirements.memory_type_bits & memory_fd_properties.memory_type_bits;
debug!(
"Memory allocation: size={}, image_type_bits={:#x}, fd_type_bits={:#x}, combined={:#x}",
mem_requirements.size,
mem_requirements.memory_type_bits,
memory_fd_properties.memory_type_bits,
memory_type_bits
);
let memory_type_index = self
.context
.find_memory_type(memory_type_bits, vk::MemoryPropertyFlags::empty())
.ok_or_else(|| {
// The numbers, in the error rather than in the debug! above
// it: fd_type_bits=0 means the driver could not resolve the
// descriptor at all, which is a different fault from a
// mismatch, and the difference is the whole diagnosis.
anyhow::anyhow!(
"No suitable memory type for DMA-BUF import: \
image_type_bits={:#x} & fd_type_bits={:#x} = {:#x}, \
size={}, format={:?}, modifier={:#x}, stride={}",
mem_requirements.memory_type_bits,
memory_fd_properties.memory_type_bits,
memory_type_bits,
mem_requirements.size,
format,
planes[0].modifier,
planes[0].stride
)
})?;
// Dedicated allocation (required by many drivers for external memory).
let mut dedicated_alloc_info = vk::MemoryDedicatedAllocateInfo::default().image(image);
import_memory_fd_info.p_next =
&mut dedicated_alloc_info as *mut vk::MemoryDedicatedAllocateInfo as *mut _;
let mut alloc_info = vk::MemoryAllocateInfo::default()
.allocation_size(mem_requirements.size)
.memory_type_index(memory_type_index);
alloc_info.p_next = &mut import_memory_fd_info as *mut vk::ImportMemoryFdInfoKHR as *mut _;
let memory = unsafe { device.allocate_memory(&alloc_info, None) }.map_err(|e| {
unsafe { device.destroy_image(image, None) };
anyhow::anyhow!("DMA-BUF memory import: {e}")
})?;
if let Err(e) = unsafe { device.bind_image_memory(image, memory, 0) } {
unsafe {
device.free_memory(memory, None);
device.destroy_image(image, None);
}
return Err(anyhow::anyhow!("DMA-BUF memory bind: {e}"));
}
Ok((image, memory))
}
}
impl Drop for DmaBufImporter {
fn drop(&mut self) {
let device = self.context.device();
unsafe {
// Clean up cached imports.
for cached in self.cached_imports.drain(..).flatten() {
device.destroy_image(cached.image, None);
device.free_memory(cached.memory, None);
}
}
}
}
apps/nescapture/src/encode.rs
+2604 -356
View File
File diff suppressed because it is too large Load Diff
apps/nescapture/src/instance.rs
+58 -8
View File
@@ -41,7 +41,42 @@ pub unsafe extern "system" fn vkCreateInstance(
let next_create: PFN_vkCreateInstance =
unsafe { load_instance_fn(next_gipa, vk::Instance::null(), b"vkCreateInstance\0") };
let result = unsafe { next_create(p_create_info, p_allocator, p_instance) };
// The encoder needs Vulkan 1.1 on the game's instance to run on the game's
// device. An application asking for 1.0 is raised to 1.1 where the loader
// has it: 1.1 only adds to 1.0, so nothing the application can do behaves
// any differently, and without it the encoder needs a device of its own.
let asked = unsafe {
let app = (*p_create_info).p_application_info;
if app.is_null() {
vk::API_VERSION_1_0
} else {
(*app).api_version
}
};
let mut raised_app;
let mut raised_ci;
let mut create_info = p_create_info;
let mut api_version = asked;
if asked < vk::API_VERSION_1_1 && unsafe { loader_version(next_gipa) } >= vk::API_VERSION_1_1 {
raised_app = unsafe {
let app = (*p_create_info).p_application_info;
if app.is_null() {
vk::ApplicationInfo::default()
} else {
*app
}
};
raised_app.api_version = vk::API_VERSION_1_1;
raised_ci = unsafe { *p_create_info };
raised_ci.p_application_info = &raised_app;
create_info = &raised_ci;
api_version = vk::API_VERSION_1_1;
log::info!(
"instance asked for Vulkan 1.0; created as 1.1 so the encoder can share its devices"
);
}
let result = unsafe { next_create(create_info, p_allocator, p_instance) };
if result != vk::Result::SUCCESS {
return result;
}
@@ -51,6 +86,8 @@ pub unsafe extern "system" fn vkCreateInstance(
// ── Build and store per-instance dispatch table ───────────────────────────
let istate = Arc::new(NextInstanceFn {
instance,
api_version,
get_instance_proc_addr: next_gipa,
destroy_instance: unsafe { load_instance_fn(next_gipa, instance, b"vkDestroyInstance\0") },
get_physical_device_memory_properties: unsafe {
@@ -60,13 +97,6 @@ pub unsafe extern "system" fn vkCreateInstance(
b"vkGetPhysicalDeviceMemoryProperties\0",
)
},
get_physical_device_format_properties2: unsafe {
crate::try_load_instance_fn(
next_gipa,
instance,
b"vkGetPhysicalDeviceFormatProperties2\0",
)
},
get_physical_device_properties: unsafe {
crate::try_load_instance_fn(next_gipa, instance, b"vkGetPhysicalDeviceProperties\0")
},
@@ -85,6 +115,26 @@ pub unsafe extern "system" fn vkCreateInstance(
vk::Result::SUCCESS
}
/// The highest instance version the loader below supports; 1.0 when it cannot
/// say, since vkEnumerateInstanceVersion is itself a 1.1 addition.
unsafe fn loader_version(next_gipa: crate::dispatch::PFN_vkGetInstanceProcAddr) -> u32 {
type Enumerate = unsafe extern "system" fn(*mut u32) -> vk::Result;
let Some(f) = (unsafe {
crate::try_load_instance_fn::<Enumerate>(
next_gipa,
vk::Instance::null(),
b"vkEnumerateInstanceVersion\0",
)
}) else {
return vk::API_VERSION_1_0;
};
let mut version = vk::API_VERSION_1_0;
if unsafe { f(&mut version) } != vk::Result::SUCCESS {
return vk::API_VERSION_1_0;
}
version
}
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkDestroyInstance(
instance: vk::Instance,
apps/nescapture/src/lib.rs
+11 -4
View File
@@ -27,20 +27,20 @@ mod config;
mod device;
mod discovery;
mod dispatch;
mod dmabuf_import;
mod encode;
mod framebuffer;
mod instance;
mod memory;
mod modifiers;
mod pacing;
mod pipeline;
mod present;
mod rate_probe;
mod shader;
mod shared;
mod slots;
mod state;
mod timing;
mod swapchain;
mod timing;
use commands::{
vkCmdBeginRenderPass, vkCmdBeginRenderingKHR, vkCmdBindPipeline, vkCmdDraw, vkCmdDrawIndexed,
@@ -48,7 +48,7 @@ use commands::{
vkCmdDrawIndirect, vkCmdDrawIndirectCount, vkCmdDrawIndirectCountKHR, vkCmdEndRenderPass,
vkCmdEndRenderingKHR,
};
use device::{vkCreateDevice, vkDestroyDevice, vkGetDeviceQueue};
use device::{vkCreateDevice, vkDestroyDevice, vkGetDeviceQueue, vkGetDeviceQueue2};
use framebuffer::{
vkAllocateCommandBuffers, vkCreateFramebuffer, vkCreateImageView, vkDestroyFramebuffer,
vkDestroyImageView, vkFreeCommandBuffers,
@@ -100,10 +100,16 @@ pub(crate) struct VkLayerDeviceLink {
pfnNextGetDeviceProcAddr: Option<PFN_vkGetDeviceProcAddr>,
}
/// Stamps the loader's dispatch data into a dispatchable object a layer
/// created itself. See [`crate::device::stamp`].
pub(crate) type PFN_vkSetDeviceLoaderData =
unsafe extern "system" fn(vk::Device, *mut c_void) -> vk::Result;
#[repr(C)]
pub(crate) union VkLayerCreateInfoU {
pub pLayerInfo: *mut VkLayerInstanceLink,
pub pDeviceLayerInfo: *mut VkLayerDeviceLink,
pub pfnSetDeviceLoaderData: Option<PFN_vkSetDeviceLoaderData>,
}
const VK_STRUCTURE_TYPE_LOADER_INSTANCE_CREATE_INFO: i32 = 47;
@@ -271,6 +277,7 @@ unsafe fn match_device_fn(name: &[u8]) -> Option<RawFn> {
b"vkCreateDevice" => Some(to_raw(vkCreateDevice as *const () as usize)),
b"vkDestroyDevice" => Some(to_raw(vkDestroyDevice as *const () as usize)),
b"vkGetDeviceQueue" => Some(to_raw(vkGetDeviceQueue as *const () as usize)),
b"vkGetDeviceQueue2" => Some(to_raw(vkGetDeviceQueue2 as *const () as usize)),
b"vkQueuePresentKHR" => Some(to_raw(vkQueuePresentKHR as *const () as usize)),
b"vkCreateShaderModule" => Some(to_raw(vkCreateShaderModule as *const () as usize)),
apps/nescapture/src/memory.rs
+6 -1
View File
@@ -43,7 +43,12 @@ pub fn pick_memory_type(types: &[MemoryType], bits: u32, want: Want) -> Option<u
match want {
Want::DeviceLocal => (0..types.len())
.find(|&i| allowed(i) && types[i].flags.contains(vk::MemoryPropertyFlags::DEVICE_LOCAL))
.find(|&i| {
allowed(i)
&& types[i]
.flags
.contains(vk::MemoryPropertyFlags::DEVICE_LOCAL)
})
.or_else(|| (0..types.len()).find(|&i| allowed(i)))
.map(|i| i as u32),
Want::HostCoherent => (0..types.len())
apps/nescapture/src/modifiers.rs
-112
View File
@@ -1,112 +0,0 @@
// ─────────────────────────────────────────────────────────────────────────────
// modifiers.rs — choosing a DRM format modifier for the capture ring
//
// The importer has always been able to take a tiled buffer: `dmabuf_import.rs`
// builds `VkImageDrmFormatModifierExplicitCreateInfoEXT` with per-plane
// layouts and creates the image with `DRM_FORMAT_MODIFIER_EXT` tiling. Only
// the producer was linear — hard-coded `ImageTiling::LINEAR` and a `modifier`
// of zero passed down with every frame — so every capture detiled a full frame
// on the write and the encoder sampled a linear image on the read.
//
// Picking the modifier is the whole of the decision and it is pure, so it is
// here and tested rather than buried in an unsafe block.
// ─────────────────────────────────────────────────────────────────────────────
/// One entry of `VkDrmFormatModifierPropertiesListEXT`, already filtered to
/// modifiers whose tiling features cover both our write and the encoder's read.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ModifierProps {
pub modifier: u64,
pub plane_count: u32,
}
/// `DRM_FORMAT_MOD_LINEAR`.
pub const LINEAR: u64 = 0;
/// `DRM_FORMAT_MOD_INVALID`, which a driver may list and which means "let the
/// driver choose" — not something to ask for explicitly.
pub const INVALID: u64 = 0x00ff_ffff_ffff_ffff;
/// Pick the modifier to allocate the capture ring with.
///
/// Single-plane only, and deliberately so. A multi-plane modifier needs an
/// offset and a stride per plane on the import side, and the export path hands
/// out one fd with one stride — so accepting one would produce an image the far
/// side reads at the wrong offsets, which arrives at the right size and frame
/// rate carrying nonsense. Prefer any real tiled modifier; fall back to linear,
/// which is what the ring used before this existed and always works.
pub fn pick_modifier(candidates: &[ModifierProps]) -> Option<ModifierProps> {
let usable = |m: &&ModifierProps| m.plane_count == 1 && m.modifier != INVALID;
candidates
.iter()
.filter(usable)
.find(|m| m.modifier != LINEAR)
.or_else(|| candidates.iter().filter(usable).find(|m| m.modifier == LINEAR))
.copied()
}
#[cfg(test)]
mod tests {
use super::*;
const TILED: u64 = 0x0200_0000_0000_0001;
#[test]
fn a_tiled_modifier_beats_linear() {
let c = [
ModifierProps { modifier: LINEAR, plane_count: 1 },
ModifierProps { modifier: TILED, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, TILED);
}
/// Order must not decide it — the driver lists them in its own order.
#[test]
fn a_tiled_modifier_wins_from_either_position() {
let c = [
ModifierProps { modifier: TILED, plane_count: 1 },
ModifierProps { modifier: LINEAR, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, TILED);
}
/// A multi-plane modifier with one exported fd would be imported at the
/// wrong plane offsets and produce a corrupt frame rather than an error.
#[test]
fn multi_plane_modifiers_are_refused() {
let c = [
ModifierProps { modifier: 0x0200_0000_0000_0002, plane_count: 2 },
ModifierProps { modifier: LINEAR, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, LINEAR);
}
/// A multi-plane tiled modifier must not beat a single-plane linear one
/// just for being tiled.
#[test]
fn tiling_does_not_excuse_a_plane_count_we_cannot_export() {
let c = [
ModifierProps { modifier: TILED, plane_count: 4 },
ModifierProps { modifier: LINEAR, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, LINEAR);
}
/// `DRM_FORMAT_MOD_INVALID` is not a modifier to ask for.
#[test]
fn the_invalid_modifier_is_never_chosen() {
let c = [ModifierProps { modifier: INVALID, plane_count: 1 }];
assert_eq!(pick_modifier(&c), None);
}
#[test]
fn nothing_usable_is_none() {
let c = [ModifierProps { modifier: TILED, plane_count: 4 }];
assert_eq!(pick_modifier(&c), None);
}
#[test]
fn an_empty_list_is_none() {
assert_eq!(pick_modifier(&[]), None);
}
}
apps/nescapture/src/present.rs
+118 -40
View File
@@ -1,5 +1,5 @@
use crate::capture;
use crate::encode::{CapturedFrame, FrameSource, PipelineConfig, PipelineHandle};
use crate::encode::{CapturedFrame, FrameSource, PipelineConfig, PipelineHandle, PresentStep};
use crate::slots::SlotGuard;
use crate::state::{DEVICE_STATE, QUEUE_TO_DEVICE_KEY};
use ash::vk::{self, Handle};
@@ -32,6 +32,7 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
// it is the game's own frame time with this layer's cost excluded — the
// three spans then partition the wall clock between presents exactly.
let entered = std::time::Instant::now();
note_present(&ds, PresentStep::InHook);
ds.frame_counter.fetch_add(1, Ordering::Relaxed);
ds.hud_detected_frame.store(false, Ordering::Relaxed);
@@ -55,9 +56,8 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
// Rewriting the wait semaphores is only well defined for a single
// swapchain. A multi-swapchain present is rare enough that passing it
// through untouched beats getting the interposition subtly wrong.
let single_swapchain = pi.swapchain_count == 1
&& !pi.p_swapchains.is_null()
&& !pi.p_image_indices.is_null();
let single_swapchain =
pi.swapchain_count == 1 && !pi.p_swapchains.is_null() && !pi.p_image_indices.is_null();
let submission = if single_swapchain {
unsafe { try_capture(&ds, queue, pi) }
@@ -68,9 +68,11 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
let down_us = std::cell::Cell::new(std::time::Duration::ZERO);
let call_down = |info: *const vk::PresentInfoKHR| match ds.fp.queue_present_khr {
Some(f) => {
note_present(&ds, PresentStep::Presenting);
let t = std::time::Instant::now();
let r = unsafe { f(queue, info) };
down_us.set(t.elapsed());
note_present(&ds, PresentStep::InHook);
r
}
None => vk::Result::ERROR_EXTENSION_NOT_PRESENT,
@@ -82,7 +84,6 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
return r;
};
// The blit consumed the application's wait semaphores, so the present waits
// on ours instead. Presenting on the originals as well would be a second
// wait on an already-consumed signal.
@@ -121,11 +122,7 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
/// `layer` is everything in this hook that is not the down-call, both sides of
/// it added together, so `gap + layer + down` accounts for the wall clock
/// between one present and the next with nothing unattributed.
fn finish(
ds: &crate::state::DeviceState,
entered: std::time::Instant,
down: std::time::Duration,
) {
fn finish(ds: &crate::state::DeviceState, entered: std::time::Instant, down: std::time::Duration) {
// Everything this hook cost, before any deliberate waiting.
let worked = std::time::Instant::now();
@@ -138,6 +135,7 @@ fn finish(
Err(_) => std::time::Duration::ZERO,
};
if !held.is_zero() {
note_present(ds, PresentStep::Holding);
std::thread::sleep(held);
}
@@ -167,12 +165,24 @@ fn finish(
if let Ok(mut last) = ds.last_present_return.lock() {
*last = Some(now);
}
note_present(ds, PresentStep::InGame);
}
/// Tell the stall watchdog where the game's present thread is. A no-op until
/// the pipeline exists.
pub fn note_present(ds: &crate::state::DeviceState, step: PresentStep) {
if let Ok(enc) = ds.encoder.lock()
&& let Some(ref h) = *enc
{
h.note_present(step);
}
}
/// Everything the present hook needs to carry from the blit to the worker.
struct Submission {
slot: SlotGuard,
present_wait: vk::Semaphore,
blit: Option<pixelforge::TimelinePoint>,
width: u32,
height: u32,
sc_fmt: vk::Format,
@@ -185,6 +195,13 @@ unsafe fn try_capture(
pi: &vk::PresentInfoKHR,
) -> Option<Submission> {
let image_index = unsafe { *pi.p_image_indices } as usize;
// An image acquired from a retired swapchain may still be presented, and
// the tracked images belong to its replacement: the same index there is a
// different image, possibly of a different size.
let presented = unsafe { *pi.p_swapchains };
if !crate::swapchain::is_current(*ds.swapchain.lock().ok()?, presented) {
return None;
}
let (sc_image, sc_fmt, sc_ext, image_count) = {
let images = ds.swapchain_images.lock().ok()?;
let fmt = *ds.swapchain_format.lock().ok()?;
@@ -196,7 +213,7 @@ unsafe fn try_capture(
};
// Gate before any GPU work is queued. A game presenting faster than the
// target would otherwise pay a full blit and DMA-BUF export for frames the
// target would otherwise pay a full blit for frames the
// encoder throws away moments later.
let present_time = std::time::Instant::now();
let admitted = match ds.frame_gate.lock() {
@@ -221,14 +238,14 @@ unsafe fn try_capture(
}
}
let app_waits: &[vk::Semaphore] = if pi.wait_semaphore_count == 0 || pi.p_wait_semaphores.is_null()
{
&[]
} else {
unsafe {
std::slice::from_raw_parts(pi.p_wait_semaphores, pi.wait_semaphore_count as usize)
}
};
let app_waits: &[vk::Semaphore] =
if pi.wait_semaphore_count == 0 || pi.p_wait_semaphores.is_null() {
&[]
} else {
unsafe {
std::slice::from_raw_parts(pi.p_wait_semaphores, pi.wait_semaphore_count as usize)
}
};
let submission = unsafe {
capture::capture_present_frame(
@@ -246,6 +263,7 @@ unsafe fn try_capture(
Some(Submission {
slot: submission.slot,
present_wait: submission.present_wait,
blit: submission.blit,
width: sc_ext.width,
height: sc_ext.height,
sc_fmt,
@@ -273,6 +291,7 @@ fn queue_for_encode(ds: &crate::state::DeviceState, submission: Submission) {
vk_colorspace: ds.swapchain_colorspace.load(Ordering::Relaxed),
present_time: submission.present_time,
slot: Some(submission.slot),
blit: submission.blit,
});
}
@@ -303,11 +322,31 @@ fn encoder_ready(ds: &crate::state::DeviceState, ds_key: usize, width: u32, heig
log::error!("no encode pipeline configuration; capture disabled");
return;
};
match PipelineHandle::new(cfg) {
// On the game's own device where it was created for that, and
// on a device of the encoder's own where it was not, or where
// adopting it fails.
let shared = ds.shared.as_ref().and_then(|s| match s.video_context() {
Ok(ctx) => Some(ctx),
Err(e) => {
log::warn!(
"could not encode on the game's device ({e}); using a device of its own"
);
None
}
});
let on_shared = shared.is_some();
match PipelineHandle::new(cfg, shared) {
Ok(h) => {
h.set_stall_probe(Box::new(move || describe_gpu(ds_key)));
// Before the handle is published: the first capture after
// it builds the ring, and must build it for this device.
ds.shared_active.store(on_shared, Ordering::Release);
if let Ok(mut enc) = ds.encoder.lock() {
*enc = Some(h);
}
if on_shared {
log::info!("encoding on the game's own device");
}
}
Err(e) => log::error!("encode pipeline: {e}"),
}
@@ -316,9 +355,13 @@ fn encoder_ready(ds: &crate::state::DeviceState, ds_key: usize, width: u32, heig
false
}
/// Wait for a frame's blit and turn its slot into something the encoder reads.
/// Turn a frame's slot into something the encoder reads.
///
/// This is the CPU handover the two devices need: pixelforge's `VkDevice`
/// On a device the encoder shares this is immediate: the slot's image is read
/// in place, and the encoder's GPU work waits on the blit's timeline point, so
/// nothing here waits at all.
///
/// Otherwise this is the CPU handover two devices need: pixelforge's `VkDevice`
/// shares no timeline with the game's, so no semaphore can bridge them and
/// somebody has to block. It used to be a thread of its own between the present
/// hook and the encoder; it is now the first thing the encoder thread does with
@@ -331,13 +374,19 @@ pub fn resolve_source(
) -> Option<FrameSource> {
let slot_index = frame.slot.as_ref()?.index();
if let Some(blit) = frame.blit {
let ring = ds.capture_ring.lock().ok()?;
let image = ring.as_ref()?.slots.get(slot_index)?.image;
return Some(FrameSource::Shared { image, blit });
}
// Copy the handles out and drop the ring lock before waiting: the present
// hook needs that lock every frame and must not queue behind a GPU wait.
let (fence, dmabuf_fd, stride, modifier, image, memory) = {
let (fence, image, memory) = {
let ring = ds.capture_ring.lock().ok()?;
ring.as_ref()
.and_then(|r| r.slots.get(slot_index))
.map(|s| (s.fence, s.dmabuf_fd, s.stride, s.modifier, s.image, s.memory))?
.map(|s| (s.fence, s.image, s.memory))?
};
let waited = unsafe { (ds.fp.wait_for_fences)(ds.raw, 1, &fence, vk::TRUE, 1_000_000_000) };
@@ -347,30 +396,59 @@ pub fn resolve_source(
}
// After the fence, so the queries have landed and `WAIT` returns at once.
if let Some(ns) = unsafe { capture::blit_gpu_time_ns(ds, slot_index) }
if let Some(ns) = unsafe { capture::blit_gpu_time_ns(ds, slot_index, true) }
&& let Ok(enc) = ds.encoder.lock()
&& let Some(ref h) = *enc
{
h.timing.blit.record(std::time::Duration::from_nanos(ns));
}
if dmabuf_fd >= 0 {
let duped = unsafe { libc::dup(dmabuf_fd) };
if duped < 0 {
log::warn!("dup of capture DMA-BUF failed — frame dropped");
return None;
}
return Some(FrameSource::DmaBuf {
fd: duped,
stride,
// The slot's own modifier. This was hard-coded to zero, which was
// true only because the producer could not ask for anything else.
modifier,
});
}
match unsafe { capture::read_frame_pixels(ds, image, memory, frame.width, frame.height) } {
Some(p) if !p.is_empty() => Some(FrameSource::Pixels(p)),
_ => None,
}
}
/// Whether the GPU has got through the capture work this layer gave it, for
/// the stall watchdog.
///
/// A blit is queued behind the game's frame on the game's own queue, so a blit
/// that never completes means that queue stopped: the game is waiting on the
/// GPU. One that did complete means the GPU finished everything in front of
/// it, and whatever the game is waiting for is not GPU work on that queue.
///
/// Never blocks. It runs exactly when something may be holding a lock forever.
fn describe_gpu(ds_key: usize) -> String {
let Some(ds) = DEVICE_STATE.get(&ds_key).map(|s| s.clone()) else {
return "device gone".into();
};
let slots = format!(
"{} of {} capture slots free",
ds.capture_slots.available(),
crate::state::CAPTURE_SLOTS
);
let Ok(ring) = ds.capture_ring.try_lock() else {
return format!("{slots}; capture ring locked");
};
let Some(ring) = ring.as_ref() else {
return format!("{slots}; no capture ring");
};
if let Some(shared) = ds.shared.as_ref()
&& !ring.blit_timeline.is_null()
{
let reached = shared
.counter(ring.blit_timeline)
.map_or("unknown".to_string(), |v| v.to_string());
return format!(
"{slots}; last blit submitted signals {}, the GPU has reached {reached}",
ring.blit_value
);
}
let pending = ring
.slots
.iter()
.filter(|s| unsafe { (ds.fp.wait_for_fences)(ds.raw, 1, &s.fence, vk::TRUE, 0) }
== vk::Result::TIMEOUT)
.count();
format!("{slots}; {pending} blits submitted and not yet complete")
}
apps/nescapture/src/rate_probe.rs
+415
View File
@@ -0,0 +1,415 @@
//! How long the encoder takes to actually reach a bitrate it was told to use.
//!
//! # Why this exists
//!
//! The bitrate controller decides how often it is worth deciding. Everything
//! else about its design follows from one number nobody has measured: the time
//! between `set_target_bitrate` and the encoder actually producing that rate.
//!
//! Simulated against a measured 1000-mile path, the difference is the whole
//! design. With the encoder settling in a quarter second, a controller sampling
//! the send queue five times a second holds the queue at 50 ms. With the
//! encoder taking a second, the same controller at any rate holds it at about
//! 1400 ms, which is unplayable, and sampling faster buys nothing at all --
//! there is no point reacting quicker than the thing being steered can move.
//!
//! So this measures it, rather than picking a control rate and hoping.
//!
//! # Shape
//!
//! Self-driving on purpose. The ladder and the dwell are fixed here so that two
//! people on two networks produce numbers that can be laid beside each other;
//! if the steps came from a person moving a slider, they would not be.
//!
//! While a sweep is running the encoder ignores bitrates from anywhere else.
//! A controller adjusting in the background would be a second hand on the same
//! dial, and the measurement would describe the argument rather than the
//! encoder.
use std::time::{Duration, Instant};
/// Targets to step through, in kbps.
///
/// Large steps and small ones, downward and upward. Downward matters most --
/// that is the direction taken under congestion, and the direction where being
/// slow costs latency rather than picture -- but an encoder can easily be quick
/// one way and slow the other, so both are here.
const LADDER: [u32; 6] = [6_000, 1_500, 6_000, 3_000, 1_000, 4_000];
/// How long to sit at each rung.
///
/// Long enough to settle and then be seen to be steady. If settling turns out
/// to take longer than this, that is itself the answer and it is reported as a
/// failure to settle rather than as a number.
const DWELL: Duration = Duration::from_secs(5);
/// Window over which the produced rate is measured.
///
/// Short enough to see a change quickly, long enough that one large frame does
/// not look like a rate. At sixty frames a second this averages about thirty.
const WINDOW: Duration = Duration::from_millis(500);
/// How close counts as arrived.
const TOLERANCE: f32 = 0.10;
/// How long it must stay inside the tolerance to count as settled, rather than
/// having passed through on the way somewhere else.
const HOLD: Duration = Duration::from_millis(500);
/// What one rung of the ladder turned out to cost.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct StepReport {
pub from_kbps: u32,
pub to_kbps: u32,
/// Milliseconds from the command to the produced rate first being inside
/// the tolerance and staying there. `None` if it never settled.
pub settle_ms: Option<u32>,
pub settle_frames: Option<u32>,
/// Produced rate over the last second of the dwell, as a fraction of the
/// target. This is the overshoot the controller has to divide out: a
/// hardware encoder asked for 1000 does not produce 1000.
pub steady_ratio: f32,
/// Keyframes during the rung.
pub keyframes: u32,
/// The largest keyframe seen, in bytes.
///
/// Reported rather than folded in, because it is a different quantity with
/// a different consumer. Settling is about the rate control finding its
/// operating point; a keyframe is a single burst handed to the transport
/// whole. The controller needs both and must not confuse them: measured at
/// 4 s GOP, a run where keyframes were counted in the rate said settling
/// took 2266 ms where the same encoder with no keyframes said 450 ms, and
/// in one case the number went *down* when keyframes were added. That is
/// not an encoder being erratic, it is a window catching an IDR.
pub keyframe_bytes: u32,
/// How long that keyframe alone occupies the link at this rung's target,
/// in milliseconds. This is the burst a queue has to absorb, and the
/// reason a queue setpoint cannot simply be set below it.
pub keyframe_ms: u32,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Phase {
/// Letting the encoder reach a steady state before the first step.
WarmUp,
Stepping,
Done,
}
pub struct RateProbe {
phase: Phase,
rung: usize,
/// When the current rung was commanded.
began: Instant,
current_kbps: u32,
/// `(when, bytes)` inside the measurement window, delta frames only.
///
/// Keyframes are deliberately absent. One IDR is worth many delta frames,
/// so a half-second window containing one reports a rate several times the
/// truth, leaves the tolerance band, and restarts the settle clock -- which
/// measures the GOP rather than the encoder.
samples: Vec<(Instant, u32)>,
keyframe_bytes: u32,
/// Frames since the current rung was commanded.
frames: u32,
keyframes: u32,
/// When the produced rate first entered the tolerance, if it still is.
inside_since: Option<Instant>,
settled: Option<(u32, u32)>,
/// The last second of the dwell, for the steady-state ratio.
steady: Vec<(Instant, u32)>,
reports: Vec<StepReport>,
}
impl RateProbe {
/// A probe that has not started stepping yet.
pub fn new(now: Instant, starting_kbps: u32) -> Self {
Self {
phase: Phase::WarmUp,
rung: 0,
began: now,
current_kbps: starting_kbps,
samples: Vec::new(),
keyframe_bytes: 0,
frames: 0,
keyframes: 0,
inside_since: None,
settled: None,
steady: Vec::new(),
reports: Vec::new(),
}
}
pub fn finished(&self) -> bool {
self.phase == Phase::Done
}
/// Whether a bitrate from elsewhere should be ignored.
pub fn owns_the_bitrate(&self) -> bool {
self.phase != Phase::Done
}
/// One encoded frame.
pub fn observe(&mut self, now: Instant, bytes: u32, keyframe: bool) {
if keyframe {
self.keyframe_bytes = self.keyframe_bytes.max(bytes);
} else {
self.samples.push((now, bytes));
}
self.samples
.retain(|(t, _)| now.duration_since(*t) <= WINDOW);
// The steady-state ratio *does* include keyframes: it answers what the
// link actually carries for a given target, which is the whole output.
self.steady.push((now, bytes));
self.steady
.retain(|(t, _)| now.duration_since(*t) <= Duration::from_secs(1));
if self.phase != Phase::Stepping {
return;
}
self.frames += 1;
if keyframe {
self.keyframes += 1;
}
if self.settled.is_some() {
return;
}
let Some(rate) = self.measured_kbps(now) else {
return;
};
let drift =
(rate as f32 - self.current_kbps as f32).abs() / self.current_kbps.max(1) as f32;
if drift > TOLERANCE {
// Left the band, so whatever it was doing was not settling.
self.inside_since = None;
return;
}
let entered = *self.inside_since.get_or_insert(now);
if now.duration_since(entered) >= HOLD {
// Credit the moment it arrived, not the moment it had stayed long
// enough to prove it: the hold is evidence about the arrival, not
// part of the time the encoder took.
let ms = entered.duration_since(self.began).as_millis() as u32;
self.settled = Some((ms, self.frames));
}
}
/// Produced rate over the window, or `None` before there is a window's worth.
fn measured_kbps(&self, now: Instant) -> Option<u32> {
let oldest = self.samples.first()?.0;
let span = now.duration_since(oldest);
if span < WINDOW / 2 {
return None;
}
let bits: u64 = self.samples.iter().map(|(_, b)| u64::from(*b) * 8).sum();
Some((bits as f64 / span.as_secs_f64() / 1000.0) as u32)
}
/// The next target to apply, when the current rung is done.
pub fn due_step(&mut self, now: Instant) -> Option<u32> {
if now.duration_since(self.began) < DWELL {
return None;
}
if self.phase == Phase::Stepping {
self.close_rung(now);
}
if self.rung >= LADDER.len() {
self.phase = Phase::Done;
return None;
}
let next = LADDER[self.rung];
self.rung += 1;
self.phase = Phase::Stepping;
self.began = now;
self.frames = 0;
self.keyframes = 0;
self.keyframe_bytes = 0;
self.inside_since = None;
self.settled = None;
let from = self.current_kbps;
self.current_kbps = next;
let _ = from;
Some(next)
}
fn close_rung(&mut self, now: Instant) {
let steady_bits: u64 = self.steady.iter().map(|(_, b)| u64::from(*b) * 8).sum();
let span = self
.steady
.first()
.map(|(t, _)| now.duration_since(*t).as_secs_f64())
.unwrap_or(0.0);
let steady_kbps = if span > 0.0 {
steady_bits as f64 / span / 1000.0
} else {
0.0
};
let from = self.reports.last().map_or(0, |r| r.to_kbps);
self.reports.push(StepReport {
from_kbps: from,
to_kbps: self.current_kbps,
settle_ms: self.settled.map(|(ms, _)| ms),
settle_frames: self.settled.map(|(_, f)| f),
steady_ratio: (steady_kbps / f64::from(self.current_kbps.max(1))) as f32,
keyframes: self.keyframes,
keyframe_bytes: self.keyframe_bytes,
keyframe_ms: (u64::from(self.keyframe_bytes) * 8 / u64::from(self.current_kbps.max(1)))
as u32,
});
}
pub fn reports(&self) -> &[StepReport] {
&self.reports
}
/// The number the controller design turns on: the slowest settle observed.
///
/// The slowest rather than the average, for the same reason the bitrate
/// controller reads the worst client's report: a loop that keeps up with
/// the typical step and not the worst one is a loop that falls behind
/// exactly when the path is changing, which is the only time it matters.
pub fn worst_settle_ms(&self) -> Option<u32> {
self.reports.iter().map(|r| r.settle_ms).max().flatten()
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Feed a probe frames at `kbps` for `secs`, sixty a second.
fn feed(probe: &mut RateProbe, start: Instant, secs: f32, kbps: u32) -> Instant {
let frames = (secs * 60.0) as u32;
let bytes = (u64::from(kbps) * 1000 / 8 / 60) as u32;
let mut now = start;
for i in 1..=frames {
// From the start each time: stepping by a rounded frame interval
// drifts, and a helper that loses 4% of every second makes a probe
// measuring seconds look broken when it is not.
now = start + Duration::from_secs_f64(f64::from(i) / 60.0);
probe.observe(now, bytes, false);
}
now
}
#[test]
fn an_encoder_that_changes_at_once_settles_at_once() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
let step = p.due_step(now).expect("first rung");
assert_eq!(step, 6_000);
now = feed(&mut p, now, 5.1, 6_000);
assert_eq!(p.due_step(now), Some(1_500));
// Produces the new rate immediately.
feed(&mut p, now, 2.0, 1_500);
let r = p.reports();
assert!(!r.is_empty());
assert!(
r[0].settle_ms.is_some_and(|ms| ms < 1_200),
"an instant encoder reported {:?}",
r[0].settle_ms
);
}
/// The case that would kill the fast-loop design, and must be visible.
#[test]
fn an_encoder_that_never_gets_there_reports_no_settle() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
p.due_step(now);
now = feed(&mut p, now, 5.1, 6_000);
assert_eq!(p.due_step(now), Some(1_500));
// Ignores the command completely.
now = feed(&mut p, now, 5.1, 6_000);
p.due_step(now);
let r = p.reports();
let step = r
.iter()
.find(|r| r.to_kbps == 1_500)
.expect("the 1500 rung");
assert_eq!(
step.settle_ms, None,
"an encoder that ignored us looked settled"
);
assert!(
step.steady_ratio > 3.0,
"steady ratio {} did not show it producing four times the target",
step.steady_ratio
);
}
/// Overshoot is the other number the controller needs, so it must be real.
#[test]
fn steady_ratio_reports_the_overshoot() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
p.due_step(now);
now = feed(&mut p, now, 5.1, 6_000);
p.due_step(now);
// Asked for 1500, produces 1875: the 25% overshoot measured on hardware.
now = feed(&mut p, now, 5.1, 1_875);
p.due_step(now);
let step = p.reports().iter().find(|r| r.to_kbps == 1_500).unwrap();
assert!(
(step.steady_ratio - 1.25).abs() < 0.1,
"steady ratio {} should be about 1.25",
step.steady_ratio
);
}
/// The confound that made a 4 s GOP look like a three-times slower
/// encoder, including making one rung appear *faster* when keyframes were
/// added -- which no encoder does, and which gave the measurement away.
#[test]
fn a_keyframe_landing_mid_window_does_not_delay_the_reported_settle() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
p.due_step(now);
now = feed(&mut p, now, 5.1, 6_000);
assert_eq!(p.due_step(now), Some(1_500));
// Settles immediately, but an IDR worth a second of bitrate lands in
// the middle of the window that is meant to prove it.
let bytes = (1_500u64 * 1000 / 8 / 60) as u32;
let base = now;
for i in 1..=320u32 {
now = base + Duration::from_secs_f64(f64::from(i) / 60.0);
let key = i == 40;
p.observe(now, if key { 1_500 * 1000 / 8 } else { bytes }, key);
}
p.due_step(now);
let step = p.reports().iter().find(|r| r.to_kbps == 1_500).unwrap();
assert!(
step.settle_ms.is_some_and(|ms| ms < 1_000),
"an encoder that settled at once reported {:?} because of one keyframe",
step.settle_ms
);
// And the burst is still reported, because the queue has to absorb it.
assert_eq!(step.keyframes, 1);
assert!(
step.keyframe_ms >= 900,
"a keyframe worth a second of bitrate reported {} ms",
step.keyframe_ms
);
}
#[test]
fn a_sweep_ends_and_does_not_step_forever() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = t0;
for _ in 0..(LADDER.len() + 2) {
now = feed(&mut p, now, 5.1, 3_000);
p.due_step(now);
}
assert!(p.finished());
assert!(
!p.owns_the_bitrate(),
"a finished sweep still holds the dial"
);
assert_eq!(p.reports().len(), LADDER.len());
}
}
apps/nescapture/src/shared.rs
+1165
View File
File diff suppressed because it is too large Load Diff
apps/nescapture/src/slots.rs
+1 -1
View File
@@ -2,7 +2,7 @@
// slots.rs — ownership of the capture ring's destination buffers
//
// A captured frame travels from the present hook, through the capture worker,
// into the encoder thread, and its DMA-BUF must not be written again until the
// into the encoder thread, and its image must not be written again until the
// encoder has finished reading it. Tracking that by hand across three threads
// is how the single-buffer version got it wrong. Instead the slot index is
// carried by a guard that returns it to the pool when it drops, wherever that
apps/nescapture/src/state.rs
+43 -20
View File
@@ -25,21 +25,10 @@ pub const CAPTURE_SLOTS: usize = 4;
pub struct CaptureSlot {
pub image: vk::Image,
pub memory: vk::DeviceMemory,
/// Exported once at allocation and duplicated per frame. -1 if the export
/// failed, which sends that frame down the CPU readback path instead.
pub dmabuf_fd: std::os::raw::c_int,
pub stride: u32,
/// The DRM format modifier the driver gave this slot's image.
///
/// Carried per slot rather than assumed, and passed to the importer, which
/// creates its side with this exact value. It used to be hard-coded to
/// `DRM_FORMAT_MOD_LINEAR` on both sides — true at the time, because the
/// producer only ever asked for linear.
pub modifier: u64,
/// Signalled when this slot's blit has finished reading the swapchain and
/// writing the slot. The capture worker waits on it before handing the
/// DMA-BUF to the encoder, which reads it from a different VkDevice and so
/// cannot be synchronised with a semaphore.
/// writing the slot. Where the encoder has a device of its own, the encoder
/// thread waits on it before reading the slot back on the CPU; a device of
/// its own shares no timeline with the game's.
pub fence: vk::Fence,
}
@@ -75,12 +64,14 @@ pub struct CaptureRing {
pub timestamp_period: f32,
/// The extent `blits` were recorded for.
///
/// The ring is kept when the swapchain shrinks — `ensure_capture_ring`
/// accepts a ring at least as large as the request — so the extent can
/// change under a ring that is not rebuilt, and the recordings have to
/// follow it even though the images do not.
/// Kept separate from `size` because a swapchain can be recreated at the
/// same extent — on a format or present-mode change — which invalidates the
/// recordings without invalidating the images.
pub blit_extent: vk::Extent2D,
pub size: (u32, u32, vk::Format),
/// Which ring this is, counting from the device's first. For the log, so a
/// resize shows up as the ring it caused.
pub generation: u64,
/// Queue family the command pool was created for. Command buffers may only
/// be submitted to a queue of the family their pool belongs to, so a
/// present arriving on a different family rebuilds the ring rather than
@@ -102,6 +93,15 @@ pub struct CaptureRing {
/// cannot reacquire it until the present that waited on this semaphore is
/// done.
pub present_wait: Vec<vk::Semaphore>,
/// Timeline semaphore every blit signals, one value higher each time, or
/// null when the encoder does not share this device.
///
/// On a shared device this is the whole handover: the frame carries the
/// value its blit signals, and the encoder's GPU work waits on it. Nothing
/// waits on the CPU.
pub blit_timeline: vk::Semaphore,
/// The value the last blit signalled.
pub blit_value: u64,
}
/// Index into [`CaptureRing::blits`] for one (swapchain image, slot) pair.
@@ -177,6 +177,17 @@ pub struct DeviceState {
pub raw: vk::Device,
pub physical_device: vk::PhysicalDevice,
pub fp: NextDeviceFn,
/// The encoder's view of this device, when it runs on it. `None` when the
/// device could not be created with what the encoder needs, in which case
/// the encoder uses a device of its own.
pub shared: Option<crate::shared::SharedDevice>,
/// Whether the encode pipeline really runs on `shared`. Set once the
/// pipeline has been built on it; until then, and for good if that fails,
/// capture works as it does for a device of the encoder's own.
pub shared_active: std::sync::atomic::AtomicBool,
/// The loader's callback for stamping dispatchable objects this layer
/// creates itself. See [`crate::device::stamp`].
pub set_loader_data: Option<crate::PFN_vkSetDeviceLoaderData>,
// Phase 1: shader / pipeline
pub shader_registry: DashMap<u64, u64>,
@@ -197,7 +208,7 @@ pub struct DeviceState {
pub hudless_memory: std::sync::Mutex<Option<vk::DeviceMemory>>,
pub hudless_size: std::sync::Mutex<(u32, u32, vk::Format)>,
// Phase 4: final-frame capture (DMA-BUF exportable)
// Phase 4: final-frame capture
pub capture_ring: std::sync::Mutex<Option<CaptureRing>>,
/// Which ring slots are free. Held separately from the ring itself so a
/// slot can be returned from the encoder thread without taking the lock
@@ -219,6 +230,9 @@ pub struct DeviceState {
pub swapchain_transfer_src: std::sync::atomic::AtomicBool,
pub frame_counter: std::sync::atomic::AtomicU64,
/// Rings built for this device so far. Names the current one; see
/// [`CaptureRing::generation`].
pub ring_generation: std::sync::atomic::AtomicU64,
// Phase 3/4: per-frame HUD detection flags
pub hud_detected_frame: std::sync::atomic::AtomicBool,
@@ -229,7 +243,6 @@ pub struct DeviceState {
// Phase 7: encode + IPC pipeline (lazy-init on first frame)
pub encoder: std::sync::Mutex<Option<PipelineHandle>>,
// ── Frame-rate throttle ───────────────────────────────────────────
/// Decides which presented frames are worth capturing. Consulted in the
/// present hook, before any GPU work is queued, so a dropped frame costs
@@ -252,6 +265,16 @@ pub struct DeviceState {
pub encoder_starting: std::sync::atomic::AtomicBool,
}
impl DeviceState {
/// The shared device, when the encoder is actually running on it.
pub fn shared_encoder(&self) -> Option<&crate::shared::SharedDevice> {
self.shared.as_ref().filter(|_| {
self.shared_active
.load(std::sync::atomic::Ordering::Acquire)
})
}
}
// ── Per-command-buffer state ──────────────────────────────────────────────────
#[derive(Default, Clone)]
apps/nescapture/src/swapchain.rs
+51 -3
View File
@@ -125,6 +125,13 @@ pub unsafe extern "system" fn vkCreateSwapchainKHR(
vk::Result::SUCCESS
}
/// Whether `swapchain` is the one this layer tracks, which is the one most
/// recently created. A swapchain retired through `oldSwapchain` stays valid
/// until destroyed, and calls on it must not touch the tracked state.
pub fn is_current(tracked: Option<vk::SwapchainKHR>, swapchain: vk::SwapchainKHR) -> bool {
swapchain != vk::SwapchainKHR::null() && tracked == Some(swapchain)
}
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkDestroySwapchainKHR(
device: vk::Device,
@@ -133,8 +140,17 @@ pub unsafe extern "system" fn vkDestroySwapchainKHR(
) {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
if let Some(ds) = DEVICE_STATE.get(&key) {
*ds.swapchain.lock().unwrap() = None;
*ds.swapchain_images.lock().unwrap() = Vec::new();
// Only the swapchain being tracked. A game recreating through
// `oldSwapchain` destroys the retired one after its replacement is
// created and its images fetched, and clearing then would leave the
// live swapchain with no images: every present after would skip
// capture, silently, until the next recreation.
let mut current = ds.swapchain.lock().unwrap();
if is_current(*current, swapchain) {
*current = None;
*ds.swapchain_images.lock().unwrap() = Vec::new();
}
drop(current);
if let Some(destroy_fn) = ds.fp.destroy_swapchain_khr {
unsafe { destroy_fn(device, swapchain, p_allocator) };
}
@@ -171,7 +187,8 @@ pub unsafe extern "system" fn vkGetSwapchainImagesKHR(
return result;
}
if !p_swapchain_images.is_null() {
// Images of a retired swapchain are not the ones capture reads.
if !p_swapchain_images.is_null() && is_current(*ds.swapchain.lock().unwrap(), swapchain) {
let count = unsafe { *p_swapchain_image_count as usize };
let images = unsafe { std::slice::from_raw_parts(p_swapchain_images, count) };
*ds.swapchain_images.lock().unwrap() = images.to_vec();
@@ -231,9 +248,11 @@ pub unsafe extern "system" fn vkAcquireNextImageKHR(
return vk::Result::ERROR_EXTENSION_NOT_PRESENT;
};
crate::present::note_present(&ds, crate::encode::PresentStep::Acquiring);
let started = std::time::Instant::now();
let result = unsafe { acquire(device, swapchain, timeout, semaphore, fence, p_image_index) };
record_acquire(&ds, started.elapsed());
crate::present::note_present(&ds, crate::encode::PresentStep::InGame);
result
}
@@ -250,8 +269,37 @@ pub unsafe extern "system" fn vkAcquireNextImage2KHR(
return vk::Result::ERROR_EXTENSION_NOT_PRESENT;
};
crate::present::note_present(&ds, crate::encode::PresentStep::Acquiring);
let started = std::time::Instant::now();
let result = unsafe { acquire(device, p_acquire_info, p_image_index) };
record_acquire(&ds, started.elapsed());
crate::present::note_present(&ds, crate::encode::PresentStep::InGame);
result
}
#[cfg(test)]
mod tests {
use super::*;
fn sc(raw: u64) -> vk::SwapchainKHR {
vk::SwapchainKHR::from_raw(raw)
}
#[test]
fn the_tracked_swapchain_is_current() {
assert!(is_current(Some(sc(2)), sc(2)));
}
#[test]
fn a_retired_swapchain_is_not() {
// Created 2 with oldSwapchain = 1; destroying 1 afterwards must leave
// 2's state alone.
assert!(!is_current(Some(sc(2)), sc(1)));
}
#[test]
fn nothing_is_current_once_the_tracked_one_is_gone() {
assert!(!is_current(None, sc(1)));
assert!(!is_current(None, vk::SwapchainKHR::null()));
}
}
apps/nescope/Cargo.toml
-2
View File
@@ -28,8 +28,6 @@ smithay = { version = "0.7", default-features = false, features = [
# Wayland client – connects to the host compositor to forward buffers.
wayland-client = "0.31"
wayland-protocols = { version = "0.32", features = ["client", "staging", "server"] }
# Needed to generate the gamescope_swapchain protocol bindings.
wayland-scanner = "0.31"
wayland-backend = "0.3"
# Event loop
apps/nescope/build.rs
-3
View File
@@ -1,3 +0,0 @@
fn main() {
println!("cargo:rerun-if-changed=src/protocols/gamescope-swapchain.xml");
}
apps/nescope/src/handlers.rs
+25 -2
View File
@@ -11,6 +11,7 @@ use smithay::desktop::Window;
use smithay::input::pointer::{CursorImageStatus, PointerHandle};
use smithay::input::{Seat, SeatHandler, SeatState};
use smithay::output::Output;
use smithay::reexports::wayland_protocols::xdg::shell::server::xdg_toplevel;
use smithay::reexports::wayland_server::protocol::wl_buffer;
use smithay::reexports::wayland_server::protocol::wl_output::WlOutput;
use smithay::reexports::wayland_server::protocol::wl_seat::WlSeat;
@@ -119,7 +120,6 @@ impl CompositorHandler for NescopeState {
fn destroyed(&mut self, surface: &WlSurface) {
self.hdr.surface_destroyed(surface);
self.vulkan_surfaces.remove(surface);
}
}
@@ -138,7 +138,7 @@ impl DmabufHandler for NescopeState {
_dmabuf: Dmabuf,
notifier: ImportNotifier,
) {
// Accept unconditionally — libhudless reads buffers
// Accept unconditionally — the nescapture layer reads buffers
// directly from the game's Vulkan queue; nescope doesn't need to.
let _ = notifier.successful::<NescopeState>();
}
@@ -187,6 +187,29 @@ impl XdgShellHandler for NescopeState {
self.determine_and_apply_focus();
}
// Granted, not merely acknowledged. The default answers with a configure
// that lacks the fullscreen state, which a client reads as a refusal: Wine
// then asks again on every window update and never treats its window as
// fullscreen, so a game switching to exclusive fullscreen -- Control does
// this on leaving its title screen -- stalls in the transition and stays
// where it was. Wine also scales an emulated display mode up to the output
// only for a fullscreen window.
fn fullscreen_request(&mut self, surface: ToplevelSurface, _output: Option<WlOutput>) {
surface.with_pending_state(|state| {
state.states.set(xdg_toplevel::State::Fullscreen);
state.size = Some((self.width as i32, self.height as i32).into());
});
surface.send_configure();
}
// Still the size of the output: there is nowhere else for a window to be.
fn unfullscreen_request(&mut self, surface: ToplevelSurface) {
surface.with_pending_state(|state| {
state.states.unset(xdg_toplevel::State::Fullscreen);
});
surface.send_configure();
}
fn new_popup(&mut self, _: PopupSurface, _: PositionerState) {}
fn grab(&mut self, _: PopupSurface, _: WlSeat, _: Serial) {}
fn reposition_request(&mut self, _: PopupSurface, _: PositionerState, _: u32) {}
apps/nescope/src/hdr.rs
+564 -286
View File
File diff suppressed because it is too large Load Diff
apps/nescope/src/input.rs
-20
View File
@@ -106,11 +106,6 @@ pub fn process_input(event: InputEvent, state: &mut NescopeState) {
_ => state.last_pointer_activity = std::time::Instant::now(),
}
// One-time X11 focus reset when the gamescope WSI surface is active.
if state.override_surface.is_some() && state.x11_focus_needs_reset {
state.sync_x11_focus();
}
match event {
InputEvent::KeyDown { keycode } => {
if let Some(kb) = state.seat.get_keyboard() {
@@ -363,21 +358,6 @@ fn clamp_cursor(state: &mut NescopeState) {
/// Find the focused target under the current cursor position.
pub fn surface_under(state: &NescopeState) -> Option<(KeyboardFocusTarget, Point<f64, Logical>)> {
if state.override_surface.is_some() {
if let Some(wid) = state.focused_x11_window {
for window in state.space.elements() {
if let Some(x11) = window.x11_surface() {
if x11.window_id() == wid {
let loc = state.space.element_geometry(window)?.loc;
return Some((KeyboardFocusTarget::Window(window.clone()), loc.to_f64()));
}
}
}
}
let (window, loc) = state.space.element_under(state.cursor_position)?;
return Some((KeyboardFocusTarget::Window(window.clone()), loc.to_f64()));
}
// Try element_under first
if let Some((window, loc)) = state.space.element_under(state.cursor_position) {
return Some((KeyboardFocusTarget::Window(window.clone()), loc.to_f64()));
apps/nescope/src/main.rs
+44 -28
View File
@@ -4,7 +4,7 @@
//!
//! nescope creates a virtual Wayland output, starts XWayland, and gives games
//! a complete compositor environment. Frames are captured externally by a
//! Vulkan interception library (`hudless`); nescope itself
//! Vulkan interception library (`nescapture`); nescope itself
//! never allocates a GBM pool or forwards DMA-BUFs.
//!
//! # Usage
@@ -17,7 +17,7 @@
//! --height <N> Output height [default: 1080]
//! --fps <N> Virtual refresh rate, advertised only [default: 60]
//! --frame-callback-hz <N> wl_surface.frame cadence [default: 1000]
//! --hdr Enable HDR protocols (wp_color_management_v1 + gamescope_swapchain)
//! --hdr Enable HDR colour management (wp_color_manager_v1)
//! --socket <NAME> Wayland socket name [default: nescope-0]
//! ```
//!
@@ -67,7 +67,6 @@ mod hdr;
mod input;
mod input_ipc;
mod libinput_backend;
mod protocols;
//mod screenshot_ipc;
//mod screenshot_wire;
mod state;
@@ -80,6 +79,22 @@ use state::{CalloopData, ClientState, NescopeState};
// CLI
// ---------------------------------------------------------------------------
/// A flag that can also arrive as an environment variable.
///
/// `--hdr` on its own still means true. The difference is what a value may be:
/// clap's own bool parser takes `true` and `false` and nothing else, so
/// `NESCOPE_HDR=1` -- which is how every other environment variable in this
/// stack is written, and the first thing anyone tries -- was rejected outright.
fn flag_value(value: &str) -> Result<bool, String> {
match value.trim().to_ascii_lowercase().as_str() {
"1" | "true" | "yes" | "on" => Ok(true),
"0" | "false" | "no" | "off" | "" => Ok(false),
other => Err(std::format!(
"expected 1 or 0 (true/false, yes/no and on/off are also taken), got {other:?}"
)),
}
}
#[derive(Parser, Debug)]
#[command(
name = "nescope",
@@ -125,8 +140,15 @@ struct Args {
#[arg(long, default_value = "1000", env = "NESCOPE_FRAME_CALLBACK_HZ")]
frame_callback_hz: u32,
/// Enable HDR protocols (wp_color_management_v1 + gamescope_swapchain_factory_v2).
#[arg(long, env = "NESCOPE_HDR")]
/// Enable HDR colour management (`wp_color_manager_v1`).
#[arg(
long,
env = "NESCOPE_HDR",
num_args = 0..=1,
default_value_t = false,
default_missing_value = "true",
value_parser = flag_value,
)]
hdr: bool,
/// Run XWayland, for Linux-native software with no Wayland support.
@@ -137,7 +159,14 @@ struct Args {
/// which is what the launch environment does -- and HDR is only offered on
/// the Wayland surface, so a game routed through XWayland loses it too.
/// Turn this on for the shrinking set of X11-only native software.
#[arg(long, env = "NESCOPE_XWAYLAND")]
#[arg(
long,
env = "NESCOPE_XWAYLAND",
num_args = 0..=1,
default_value_t = false,
default_missing_value = "true",
value_parser = flag_value,
)]
xwayland: bool,
/// Wayland socket name (created in $XDG_RUNTIME_DIR).
@@ -422,8 +451,7 @@ fn main() {
// wrong way round, and it capped them at 60 while sessions asked for 120.
// The capture layer holds the game instead, and this runs fast enough to
// stay out of the way.
let frame_interval =
Duration::from_micros(1_000_000 / args.frame_callback_hz.max(1) as u64);
let frame_interval = Duration::from_micros(1_000_000 / args.frame_callback_hz.max(1) as u64);
loop_handle
.insert_source(Timer::from_duration(frame_interval), move |_, _, data| {
if let Some(ref mut li) = data.libinput {
@@ -437,7 +465,7 @@ fn main() {
// ── CalloopData ───────────────────────────────────────────────────────
let socket_name_for_cleanup = args.socket.clone();
let command = args.command.clone();
let gamescope_wayland_socket = args.socket.clone();
let wayland_socket = args.socket.clone();
// ── libinput backend ─────────────────────────────────────────────────
let libinput_ctx =
@@ -491,7 +519,7 @@ fn main() {
// Put the game in its own process group so we can
// kill the whole tree at once with kill(-pgid, …).
.process_group(0)
.env("WAYLAND_DISPLAY", &gamescope_wayland_socket);
.env("WAYLAND_DISPLAY", &wayland_socket);
// DISPLAY only if XWayland is actually running. Setting it
// otherwise points clients at a server that is not there,
@@ -516,24 +544,6 @@ fn main() {
// this. Without it neither DX11 nor DX12 (vkd3d-proton
// through DXVK's dxgi) sees HDR as available.
cmd.env("DXVK_HDR", "1");
// Left set, but deliberately without ENABLE_GAMESCOPE_WSI
// alongside it, so it is inert unless somebody opts in.
//
// That pair activates gamescope's WSI layer, which
// predates Wayland colour management and works by
// hiding HDR from the driver and reporting it to the
// compositor out of band. We do not want it: it needs a
// layer this image does not ship, it only helps the
// XWayland path, and capture reads the colour space it
// hides -- measured, a game asking for HDR10 through it
// has its PQ samples encoded and tagged BT.709 SDR.
// Enabling it would trade no HDR for wrong HDR.
tracing::debug!(
gamescope_wayland_socket,
"HDR: Wayland colour management; gamescope WSI not enabled"
);
cmd.env("GAMESCOPE_WAYLAND_DISPLAY", &gamescope_wayland_socket);
}
// Detect GPU vendor from render device and set VK_DRIVER_FILES
@@ -625,6 +635,12 @@ fn main() {
}
}
// Answer any colour-management information requests that came in
// this iteration. Deferred to here because the event that ends
// them destroys the object, and doing that inside the request that
// created it panics the backend -- see .
data.state.hdr.flush_information();
// ── Flush Wayland clients ─────────────────────────────────
if let Err(e) = data.display.flush_clients() {
tracing::warn!("Error flushing Wayland clients: {e}");
apps/nescope/src/protocols/gamescope-swapchain.xml
-194
View File
@@ -1,194 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="gamescope_swapchain">
<copyright>
Copyright © 2023 Joshua Ashton for Valve Software
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
</copyright>
<description summary="gamescope-specific swapchain protocol">
This is a private Gamescope protocol. Regular Wayland clients must not use
it.
</description>
<interface name="gamescope_swapchain_factory_v2" version="1">
<request name="destroy" type="destructor"></request>
<request name="create_swapchain">
<description summary="create Gamescope swapchain interface">
</description>
<arg name="surface" type="object" interface="wl_surface"
summary="target surface"/>
<arg name="callback" type="new_id" interface="gamescope_swapchain"
summary="new swapchain object"/>
</request>
</interface>
<interface name="gamescope_swapchain" version="1">
<request name="destroy" type="destructor"></request>
<request name="override_window_content">
<description summary="override an X11's window wl_surface">
Xwayland creates a wl_surface for each X11 window. It sends a
WL_SURFACE_ID client message to indicate the mapping between the X11
windows and the wl_surface objects.
This request overrides this mapping for a given X11 window, allowing an
X11 client to submit buffers via the Wayland protocol. The override
only affects buffer submission. Everything else (e.g. input events)
still uses Xwayland's WL_SURFACE_ID.
x11_server is gotten by the GAMESCOPE_XWAYLAND_SERVER_ID property on the
root window of the associated server.
</description>
<arg name="gamescope_xwayland_server_id" type="uint" summary="gamescope xwayland server ID"/>
<arg name="x11_window" type="uint" summary="X11 window ID"/>
</request>
<request name="swapchain_feedback">
<description summary="provide swapchain feedback">
Provide swapchain feedback to the compositor.
This is what the useless tearing protocol should have been.
Absolutely not enough information in the final protocol to do what we want for SteamOS --
which is have the Allow Tearing toggle apply to *both* Mailbox + Immediate and NOT fifo,
essentially acting as an override for tearing on/off for games.
The upstream protocol is very useless for our usecase here.
Provides image count ahead of time instead of needing to try and calculate it from
an initial stall if we are doing low latency.
Provides colorspace info for us to do HDR for both HDR10 PQ and scRGB.
The upstream HDR efforts seem to have no interest in supporting scRGB but we *need* that so /shrug
We can do it here now! Yipee!
Swapchain feedback solves so many problems! :D
</description>
<arg name="image_count" type="uint" summary="image count of swapchain"/>
<arg name="vk_format" type="uint" summary="VkFormat of swapchain"/>
<arg name="vk_colorspace" type="uint" summary="VkColorSpaceKHR of swapchain"/>
<arg name="vk_composite_alpha" type="uint" summary="VkCompositeAlphaFlagBitsKHR of swapchain"/>
<arg name="vk_pre_transform" type="uint" summary="VkSurfaceTransformFlagBitsKHR of swapchain"/>
<arg name="vk_clipped" type="uint" summary="clipped (VkBool32) of swapchain"/>
<arg name="vk_engine_name" type="string" summary="Engine name"/>
</request>
<request name="set_present_mode">
<description summary="Add a fifo queue constraint"/>
<arg name="vk_present_mode" type="uint" summary="VkPresentModeKHR"/>
</request>
<request name="set_hdr_metadata">
<description summary="set HDR metadata for a surface">
Forward HDR metadata from Vulkan to the compositor.
HDR Metadata Infoframe as per CTA 861.G spec.
This is expected to match exactly with the spec.
display_primary_*:
Color Primaries of the Data.
Specifies X and Y coordinates.
These are coded as unsigned 16-bit values in units of
0.00002, where 0x0000 represents zero and 0xC350
represents 1.0000.
white_point_*:
White Point of Colorspace Data.
Specifies X and Y coordinates.
These are coded as unsigned 16-bit values in units of
0.00002, where 0x0000 represents zero and 0xC350
represents 1.0000.
max_display_mastering_luminance:
Max Mastering Display Luminance.
This value is coded as an unsigned 16-bit value in units of 1 cd/m2,
where 0x0001 represents 1 cd/m2 and 0xFFFF represents 65535 cd/m2.
max_display_mastering_luminance:
Min Mastering Display Luminance.
This value is coded as an unsigned 16-bit value in units of
0.0001 cd/m2, where 0x0001 represents 0.0001 cd/m2 and 0xFFFF
represents 6.5535 cd/m2.
max_cll:
Max Content Light Level.
This value is coded as an unsigned 16-bit value in units of 1 cd/m2,
where 0x0001 represents 1 cd/m2 and 0xFFFF represents 65535 cd/m2.
max_fall:
Max Frame Average Light Level.
This value is coded as an unsigned 16-bit value in units of 1 cd/m2,
where 0x0001 represents 1 cd/m2 and 0xFFFF represents 65535 cd/m2.
</description>
<arg name="display_primary_red_x" type="uint" summary="red primary x coordinate"/>
<arg name="display_primary_red_y" type="uint" summary="red primary y coordinate"/>
<arg name="display_primary_green_x" type="uint" summary="green primary x coordinate"/>
<arg name="display_primary_green_y" type="uint" summary="green primary y coordinate"/>
<arg name="display_primary_blue_x" type="uint" summary="blue primary x coordinate"/>
<arg name="display_primary_blue_y" type="uint" summary="blue primary y coordinate"/>
<arg name="white_point_x" type="uint" summary="white point x coordinate"/>
<arg name="white_point_y" type="uint" summary="white point y coordinate"/>
<arg name="max_display_mastering_luminance" type="uint" summary="max display mastering luminance"/>
<arg name="min_display_mastering_luminance" type="uint" summary="min display mastering luminance"/>
<arg name="max_cll" type="uint" summary="max content light level"/>
<arg name="max_fall" type="uint" summary="max frame average light level"/>
</request>
<request name="set_present_time">
<description summary="display timing of next commit">
Sets the display timing of the next commit.
This gets reset to 0s in the compositor's state after a commit.
</description>
<arg name="present_id" type="uint" summary="application provided presentation id"/>
<arg name="desired_present_time_hi" type="uint" summary="high part of the desired presentation time for this commit. Uses CLOCK_MONOTONIC. 0 = present as soon as possible."/>
<arg name="desired_present_time_lo" type="uint" summary="low part of the desired presentation time for this commit. Uses CLOCK_MONOTONIC. 0 = present as soon as possible."/>
</request>
<event name="past_present_timing">
<description summary="information about past presentation">
Gives information on the past presentation timing
</description>
<arg name="present_id" type="uint" summary="application provided presentation id"/>
<arg name="desired_present_time_hi" type="uint" summary="high part of the desired presentation time for the commit. (from the app)"/>
<arg name="desired_present_time_lo" type="uint" summary="low part of the desired presentation time for the commit. (from the app)"/>
<arg name="actual_present_time_hi" type="uint" summary="high part of the actual present time for this commit."/>
<arg name="actual_present_time_lo" type="uint" summary="low part of the actual present time for this commit."/>
<arg name="earliest_present_time_hi" type="uint" summary="high part of the refresh time that Gamescope thought this commit was done by."/>
<arg name="earliest_present_time_lo" type="uint" summary="low part of the refresh time that Gamescope thought this commit was done by."/>
<arg name="present_margin_hi" type="uint" summary="high part of the difference between earliest present time and the earliest latch time"/>
<arg name="present_margin_lo" type="uint" summary="low part of the difference between earliest present time and the earliest latch time"/>
</event>
<event name="refresh_cycle">
<description summary="information about refresh cycle for this swapchain">
Gives information on the refresh cycle for this swapchain
</description>
<arg name="refresh_cycle_hi" type="uint" summary="high part of the refresh cycle in nanos"/>
<arg name="refresh_cycle_lo" type="uint" summary="low part of the refresh cycle in nanos"/>
</event>
<event name="retired">
<description summary="Swapchain was remotely retired"></description>
</event>
</interface>
</protocol>
apps/nescope/src/protocols/mod.rs
-16
View File
@@ -1,16 +0,0 @@
//! Generated Wayland protocol bindings for the gamescope swapchain protocol.
#![allow(non_upper_case_globals, non_camel_case_types, unused)]
use smithay::reexports::wayland_server;
use wayland_server::protocol::*;
pub mod __interfaces {
use super::wayland_server;
use wayland_server::backend as wayland_backend;
use wayland_server::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("src/protocols/gamescope-swapchain.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_server_code!("src/protocols/gamescope-swapchain.xml");
apps/nescope/src/state.rs
+27 -41
View File
@@ -24,6 +24,7 @@ use calloop::channel::Sender;
use smithay::desktop::utils::{
OutputPresentationFeedback, send_frames_surface_tree,
surface_presentation_feedback_flags_from_states, surface_primary_scanout_output,
take_presentation_feedback_surface_tree,
};
use smithay::desktop::{Space, Window};
use smithay::input::pointer::CursorImageStatus;
@@ -96,13 +97,18 @@ impl ClientData for ClientState {
// X11 atoms + connection
// ---------------------------------------------------------------------------
/// Gamescope-compatible X11 atoms used for focus and HDR signalling.
/// Gamescope-compatible X11 atoms, for focus.
///
/// Focus only. HDR used to be signalled through one of these as well, back
/// when a WSI layer inside the game reported the colour space out of band;
/// that route is gone and `wp_color_manager_v1` carries it. These remain
/// because Steam reads them to work out which window it is looking at, and
/// that is unrelated to colour.
pub struct CachedAtoms {
pub net_active_window: u32,
pub gamescope_focused_app: u32,
pub gamescope_focusable_apps: u32,
pub gamescope_focusable_windows: u32,
pub gamescope_hdr_output_feedback: u32,
pub gamescope_xwayland_server_id: u32,
pub xa_window: u32,
pub xa_cardinal: u32,
@@ -157,10 +163,6 @@ pub struct NescopeState {
pub focused_app_id: u32,
/// True when X11 focus needs to be re-synced on the next input event.
pub x11_focus_needs_reset: bool,
/// Gamescope WSI override surface (direct Vulkan → Wayland bypass).
pub override_surface: Option<WlSurface>,
/// Surfaces that have announced themselves as Vulkan via gamescope protocol.
pub vulkan_surfaces: HashSet<WlSurface>,
// ── Input ─────────────────────────────────────────────────────────────
/// Sender half of the input channel — clone and hand to callers.
@@ -267,7 +269,7 @@ impl NescopeState {
let (dmabuf_state, dmabuf_global) =
build_dmabuf_global::<Self>(&display_handle, render_device.as_deref());
// HDR + gamescope swapchain globals (optional).
// Colour management, when asked for.
let hdr_state = HdrState::new(&display_handle, hdr);
// Input channel — the Sender is returned to the caller.
@@ -309,8 +311,6 @@ impl NescopeState {
focused_x11_window: None,
focused_app_id: 0,
x11_focus_needs_reset: false,
override_surface: None,
vulkan_surfaces: HashSet::new(),
input_tx: input_tx.clone(),
cursor_position: Point::from((0.0f64, 0.0f64)),
cursor_status: CursorImageStatus::default_named(),
@@ -398,10 +398,6 @@ impl NescopeState {
gamescope_focused_app: intern_atom(&conn, b"GAMESCOPE_FOCUSED_APP"),
gamescope_focusable_apps: intern_atom(&conn, b"GAMESCOPE_FOCUSABLE_APPS"),
gamescope_focusable_windows: intern_atom(&conn, b"GAMESCOPE_FOCUSABLE_WINDOWS"),
gamescope_hdr_output_feedback: intern_atom(
&conn,
b"GAMESCOPE_HDR_OUTPUT_FEEDBACK",
),
gamescope_xwayland_server_id: intern_atom(
&conn,
b"GAMESCOPE_XWAYLAND_SERVER_ID",
@@ -422,8 +418,10 @@ impl NescopeState {
}
}
/// Write gamescope-specific X11 root window properties so the WSI layer
/// can discover this compositor as a gamescope-compatible server.
/// Write the gamescope-compatible X11 root window properties.
///
/// These say which application has focus and what could take it, which is
/// what Steam looks for. Nothing here concerns colour.
pub fn set_gamescope_atoms(
&self,
conn: &smithay::reexports::x11rb::rust_connection::RustConnection,
@@ -438,14 +436,6 @@ impl NescopeState {
let replace = PropMode::REPLACE;
let cardinal = AtomEnum::CARDINAL;
// HDR output feedback — set to 1 when HDR is active.
let _ = conn.change_property32(
replace,
root,
atoms.gamescope_hdr_output_feedback,
cardinal,
&[1u32],
);
// XWayland server ID — always 0 for a standalone compositor.
let _ = conn.change_property32(
replace,
@@ -465,16 +455,6 @@ impl NescopeState {
tracing::debug!("Set gamescope atoms on display :{display_number}");
}
// -----------------------------------------------------------------------
// Override surface (gamescope WSI bypass)
// -----------------------------------------------------------------------
/// Register the gamescope WSI override surface for an X11 window.
pub fn override_window_surface(&mut self, x11_window: u32, surface: WlSurface) {
tracing::debug!(x11_window, "Registered gamescope WSI override surface");
self.override_surface = Some(surface);
}
// -----------------------------------------------------------------------
// Resize
// -----------------------------------------------------------------------
@@ -769,12 +749,24 @@ impl NescopeState {
// 1. Release the held buffer → frees a swapchain image for the game.
self.held_buffer.take();
// 2. Presentation feedback — tell clients about vsync timing.
// 2. Presentation feedback — every frame committed since the last tick
// is reported presented, on the one output there is.
//
// Not filtered by primary scan-out output: that is recorded by a
// renderer, and nothing here renders, so the filter matched no
// surface ever. Feedback then resolved only as `discarded`, when the
// next commit superseded it -- which never happens for a client that
// waits for its last present before drawing the next. A Vulkan
// client with present-wait under FIFO does exactly that: Control on
// VKD3D-Proton froze on leaving its title screen, GPU idle, the game
// still running behind a stream that no longer moved.
let mut output_presentation_feedback = OutputPresentationFeedback::new(&output);
let on_output =
|_: &WlSurface, _: &smithay::wayland::compositor::SurfaceData| Some(output.clone());
for window in self.space.elements().cloned().collect::<Vec<_>>() {
window.take_presentation_feedback(
&mut output_presentation_feedback,
surface_primary_scanout_output,
on_output,
|_, _| wp_presentation_feedback::Kind::Vsync,
);
}
@@ -797,12 +789,6 @@ impl NescopeState {
}
}
if let Some(ref s) = self.override_surface {
send_frames_surface_tree(s, &output, now, Some(Duration::ZERO), |_, _| {
Some(output.clone())
});
}
// 4. Send periodic stats over IPC
let now = std::time::Instant::now();
if now.duration_since(self.last_stats_time) >= std::time::Duration::from_secs(1) {
apps/neshub/Cargo.toml
+1 -1
View File
@@ -11,7 +11,7 @@ name = "neshub"
path = "src/main.rs"
[dependencies]
iroh = "1.1.0"
iroh = "1.2"
anyhow.workspace = true
clap.workspace = true
apps/neshub/src/control.rs
+914
View File
@@ -0,0 +1,914 @@
//! Deciding what the encoder may spend.
//!
//! # Why this exists at all
//!
//! Nothing used to decide. The encoder took a bitrate from an environment
//! variable nobody set, so every session offered the same 10 Mbps whatever the
//! path between the two ends could carry. On a path sustaining under 3 Mbps that
//! does not degrade the picture, it removes it: most fragments are dropped,
//! every frame made of several fragments arrives incomplete, and the keyframe --
//! the largest frame there is -- never completes either, so nothing ever
//! recovers.
//!
//! # Why the receiver decides, and not this end
//!
//! The obvious inputs are here: the hub holds the QUIC connection and can read
//! its round-trip time, its congestion window and its remaining datagram buffer.
//! They were measured, over an entire multi-session run, saying the path was
//! healthy while the client was receiving almost nothing -- 182 ms round trip,
//! *zero* QUIC packet loss, and arrivals falling from 1776 to 239 datagrams a
//! second.
//!
//! Part of that is structural rather than bad luck. `send_datagram` evicts the
//! oldest queued datagrams and returns `Ok`, so a sender that is overrunning the
//! path is told nothing at all; the "frames dropped" counter it feeds cannot be
//! non-zero however badly things are going. So the primary evidence is the
//! receiver's own report of what arrived, and this end's view is the fallback
//! for when no report has come in.
//!
//! # Shape
//!
//! Slow and dull on purpose. This is not a congestion controller in the QUIC
//! sense and does not try to be: it finds the right order of magnitude for a
//! video bitrate and stays there, at one decision a second. The transport
//! underneath is still doing real congestion control on its own packets.
use nesprotocol::{ControlMode, ReceiverReport};
/// Loss above this means the path is being overrun and the target comes down.
const LOSS_DECREASE: f32 = 0.10;
/// Loss below this means there is room to climb.
const LOSS_INCREASE: f32 = 0.02;
/// What to keep of the measured goodput when backing off.
///
/// Backing off to *below* what actually got through, rather than to a fraction
/// of what we were asking for, is what makes recovery quick. At 10 Mbps into a
/// 3 Mbps path the next target is about 2.5 Mbps rather than 8.5, so one step
/// does what twenty multiplicative decreases would.
const BACKOFF: f32 = 0.85;
/// How much of the ceiling to add per calm second. Twenty steps from floor to
/// ceiling: slow enough that a brief quiet spell does not undo a back-off.
const CLIMB_FRACTION: u32 = 20;
/// Don't actuate for less than this fraction of the current target.
///
/// Retuning is cheap now -- it no longer rebuilds the encoder or forces a
/// keyframe -- but a line in the log every second saying the bitrate moved by
/// 1% is noise that hides the changes that matter.
const DEADBAND: f32 = 0.10;
/// Never go below this, whatever the ceiling is.
///
/// Under this a 1080p stream is not worth sending and the right answer is fewer
/// pixels rather than fewer bits, which is a decision about the tier and not one
/// this can make. It also has to leave room for audio, which shares the
/// connection and is not counted against the video ceiling.
const ABSOLUTE_FLOOR_KBPS: u32 = 800;
/// Seconds without a receiver report before this end's own view is used instead.
const SILENT_TICKS_BEFORE_FALLBACK: u32 = 3;
/// Backlog, in milliseconds of video, that means the path is being overrun.
///
/// Not a tuned number: it is a latency budget. At sixty frames a second a frame
/// is under 17 ms, so a quarter second of backlog is fifteen frames already
/// handed over and not yet gone -- a player is looking at a picture from before
/// their last four keypresses. There is no bitrate worth that, so past this the
/// target comes down whatever the loss says.
const QUEUE_DECREASE_MS: u32 = 250;
/// Backlog, in milliseconds, under which the path is considered clear.
///
/// Climbing needs a stronger warrant than holding does, because climbing is
/// what digs the queue. Five frames or so of backlog is the most that can be
/// outstanding and still be called live.
const QUEUE_CLIMB_MS: u32 = 80;
/// The band a target must stay inside.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Limits {
pub ceiling_kbps: u32,
}
impl Limits {
pub fn new(ceiling_kbps: u32) -> Self {
Self { ceiling_kbps }
}
/// The lowest this may go.
pub fn floor_kbps(&self) -> u32 {
ABSOLUTE_FLOOR_KBPS
.max(self.ceiling_kbps / 10)
// A ceiling below the floor is a tier nobody should have configured,
// but clamping the wrong way round would put the target *above* the
// ceiling, which is the one thing a ceiling must never allow.
.min(self.ceiling_kbps)
}
fn clamp(&self, kbps: u32) -> u32 {
kbps.clamp(self.floor_kbps(), self.ceiling_kbps)
}
}
/// What this end can see about the path, for when the far end is not talking.
///
/// Every field is optional because every one of them can be unavailable: a
/// connection with no established path has no round-trip time, and a congestion
/// window is only meaningful once something has been sent.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct PathView {
/// Congestion window in bytes, as the transport currently believes it.
pub cwnd_bytes: Option<u64>,
/// Round-trip time in milliseconds.
pub rtt_ms: Option<u32>,
/// Datagram bytes handed to the transport that have not yet left.
///
/// The backlog is the one thing this end can see that says the path is
/// being overrun *while it is still only late*. Loss says so too, but only
/// afterwards, and on a path that queues rather than drops, "afterwards"
/// can be several seconds.
pub backlog_bytes: Option<u64>,
}
impl PathView {
/// How long the backlog takes to drain at `drain_kbps`, in milliseconds.
///
/// Bits divided by kilobits-per-second is milliseconds. The rate to divide
/// by is the one the queue actually drains at -- what is getting through --
/// and not what we are asking for, which is the number that is too high
/// whenever this matters.
pub fn backlog_ms(&self, drain_kbps: u32) -> Option<u32> {
let backlog = self.backlog_bytes?;
let drain = u64::from(drain_kbps.max(1));
Some((backlog.saturating_mul(8) / drain).min(u64::from(u32::MAX)) as u32)
}
/// Delivery rate the window and round trip imply, in kbps.
///
/// A window is a quantity of bytes in flight for one round trip, so the two
/// together are a rate. Treated as an estimate of last resort: it describes
/// the transport's own opinion of the path, and that opinion is exactly what
/// was observed to be wrong when this mattered most.
pub fn estimate_kbps(&self) -> Option<u32> {
let (cwnd, rtt) = (self.cwnd_bytes?, self.rtt_ms?.max(1));
let bits_per_second = cwnd.saturating_mul(8) * 1000 / u64::from(rtt);
Some((bits_per_second / 1000).min(u64::from(u32::MAX)) as u32)
}
}
/// Why the target is what it is, for the overlay and the log.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[repr(u8)]
pub enum Reason {
/// Loss was high; backed off to under what was actually getting through.
Congested,
/// Loss was in the middle band; left alone.
Holding,
/// Loss was low; climbing towards the ceiling.
Climbing,
/// No receiver report recently; using this end's view of the path.
Fallback,
/// No report and no usable view either; decaying towards the floor.
Blind,
/// Somebody set the bitrate by hand.
Manual,
/// The encoder is not under a bitrate at all, so there is nothing to decide.
ConstantQuality,
/// Nobody is connected, so there is no path to have an opinion about.
NoClients,
/// This second's numbers describe frames the hub chose not to send, so they
/// say nothing about the path.
SelfInflicted,
/// Nothing was lost, but the send queue is standing deep enough that the
/// picture is arriving late. Backed off on the queue rather than on loss.
Backlogged,
}
/// The controller's whole state.
#[derive(Debug, Clone, Copy)]
pub struct Controller {
/// The ceiling the box was given, which a client may lower but never raise.
///
/// Kept apart from `limits`, which holds whatever is in force *now*. Folding
/// the two together means a client that lowers the ceiling can never raise
/// it again, because the only number left to compare against is the one it
/// just lowered.
box_ceiling_kbps: u32,
/// What the controller currently believes the bitrate should be.
///
/// Kept apart from `sent_kbps` deliberately. Folding the two together looks
/// simpler and stalls the climb: the step up is a fraction of the *ceiling*,
/// so once the target passes half of it each step is under the deadband, and
/// a target that only moved when it actuated could never accumulate past
/// that point. Belief moves every second; only saying so is rationed.
target_kbps: u32,
/// What the encoder was last told, or `None` before anything was sent.
sent_kbps: Option<u32>,
limits: Limits,
mode: ControlMode,
/// Set when the encoder is in constant-QP; there is no bitrate to control.
constant_quality: bool,
silent_ticks: u32,
reason: Reason,
/// Last measured send-queue depth, kept for reporting rather than control.
backlog_ms: u32,
}
impl Controller {
/// Start at the ceiling.
///
/// Optimistic on purpose: the common case is a path that can carry the tier
/// that was sold, and a session that opened at its floor and climbed would
/// take twenty seconds to look like what was paid for. The first bad report
/// undoes it in one step.
pub fn new(limits: Limits) -> Self {
Self {
box_ceiling_kbps: limits.ceiling_kbps,
target_kbps: limits.ceiling_kbps,
sent_kbps: None,
limits,
mode: ControlMode::Auto,
constant_quality: false,
silent_ticks: 0,
reason: Reason::Holding,
backlog_ms: 0,
}
}
pub fn target_kbps(&self) -> u32 {
self.target_kbps
}
pub fn limits(&self) -> Limits {
self.limits
}
pub fn mode(&self) -> ControlMode {
self.mode
}
pub fn reason(&self) -> Reason {
self.reason
}
/// How far behind the send queue was at the last decision, in milliseconds.
pub fn backlog_ms(&self) -> u32 {
self.backlog_ms
}
/// The ceiling the box was given, whatever a client has since asked for.
pub fn box_ceiling_kbps(&self) -> u32 {
self.box_ceiling_kbps
}
/// Adopt a ceiling a client asked for, never above the one the box was given.
///
/// A client may lower its own ceiling -- to test a path, or because it knows
/// something about its link that this end does not -- but it may not raise
/// the one the tier bought.
pub fn set_ceiling(&mut self, ceiling_kbps: u32) {
self.limits = Limits::new(ceiling_kbps.min(self.box_ceiling_kbps).max(1));
self.target_kbps = self.limits.clamp(self.target_kbps);
// A ceiling that moved is worth restating even when the target did not,
// because the encoder is the thing that has to hear about it.
self.sent_kbps = None;
}
pub fn set_mode(&mut self, mode: ControlMode) {
self.mode = mode;
if mode == ControlMode::Manual {
self.reason = Reason::Manual;
}
}
/// Note that somebody set the bitrate by hand.
pub fn note_manual_target(&mut self, kbps: u32) {
self.target_kbps = kbps;
self.sent_kbps = Some(kbps);
self.mode = ControlMode::Manual;
self.reason = Reason::Manual;
}
/// Note that the encoder is, or is no longer, under constant quality.
pub fn set_constant_quality(&mut self, constant_quality: bool) {
self.constant_quality = constant_quality;
}
/// One second's decision.
///
/// `report` is the worst report across the clients attached, or `None` when
/// none of them said anything. Returns the new target when it is worth
/// sending, and `None` when nothing should be sent -- which is most seconds.
/// One second's decision.
///
/// `self_inflicted` says this second contained frames the hub deliberately
/// withheld -- a client resynchronising, whose deltas were undecodable and
/// were dropped rather than sent. Those seconds cannot be read as evidence
/// about the path: fewer frames were sent, so fewer arrived, so the measured
/// goodput is low and the loss is high, and a controller anchoring a backoff
/// on that would cut the bitrate in response to its own decision. Every
/// stutter would then also cost bandwidth, which is the opposite of what a
/// resynchronisation needs.
pub fn tick(
&mut self,
clients: usize,
report: Option<ReceiverReport>,
path: PathView,
self_inflicted: bool,
) -> Option<u32> {
if clients == 0 {
// Nothing is connected, so nothing is being carried and there is no
// path to form an opinion about. Deciding here means deciding on the
// absence of evidence: the controller used to read it as silence and
// decay to the floor, so a box waiting for its first client spent
// that time winding itself down and then jumped back up the moment
// somebody arrived.
self.reason = Reason::NoClients;
self.silent_ticks = 0;
return None;
}
if self.constant_quality {
self.reason = Reason::ConstantQuality;
return None;
}
if self.mode == ControlMode::Manual {
self.reason = Reason::Manual;
return None;
}
if self_inflicted {
// Held, not decayed. The path may be fine; this second simply
// cannot say, and silence about the path is not evidence against it.
self.reason = Reason::SelfInflicted;
self.silent_ticks = 0;
return None;
}
// What the queue drains at is what is getting through. With no report
// to say, the target is the best guess available -- and a target that
// is too high only makes the backlog look shorter than it is, so this
// errs towards patience rather than towards cutting the rate.
let drain_kbps = match report.as_ref().map(|r| (r.goodput_bps / 1000) as u32) {
Some(measured) if measured > 0 => measured,
_ => self.target_kbps,
};
self.backlog_ms = path.backlog_ms(drain_kbps).unwrap_or(0);
let next = match report.as_ref().and_then(|r| r.loss().map(|l| (r, l))) {
Some((report, loss)) => {
self.silent_ticks = 0;
self.decide_from_report(report, loss, path)
}
None => {
self.silent_ticks = self.silent_ticks.saturating_add(1);
if self.silent_ticks < SILENT_TICKS_BEFORE_FALLBACK {
self.reason = Reason::Holding;
return None;
}
self.decide_from_path(path)
}
};
self.target_kbps = self.limits.clamp(next);
// Every second moves the belief; only a change worth acting on is said.
// The first decision is always said -- the encoder started at whatever
// its environment gave it, and this end has no way to know it matches.
match self.sent_kbps {
Some(sent) if !worth_sending(sent, self.target_kbps, self.limits) => None,
_ => {
self.sent_kbps = Some(self.target_kbps);
Some(self.target_kbps)
}
}
}
fn decide_from_report(&mut self, report: &ReceiverReport, loss: f32, path: PathView) -> u32 {
let measured = (report.goodput_bps / 1000) as u32;
// `backlog_ms` is zero both when the queue is empty and when the
// transport cannot say, which are the same thing for this purpose: a
// queue nobody can see is not evidence of one.
let known = path.backlog_bytes.is_some();
let queued = known && self.backlog_ms > QUEUE_DECREASE_MS;
if loss > LOSS_DECREASE || queued {
self.reason = if queued {
Reason::Backlogged
} else {
Reason::Congested
};
// Anchored on what actually arrived, not on what we were asking for.
// `goodput` of zero means nothing completed at all, in which case
// there is no measurement to anchor to and the target is simply
// halved -- the alternative, backing off to zero, would take the
// stream below the floor on a single bad second.
let anchor = if measured == 0 {
self.target_kbps / 2
} else {
measured.min(self.target_kbps)
};
return (anchor as f32 * BACKOFF) as u32;
}
// Climbing is what digs the queue, so it needs the queue to be empty as
// well as the loss to be low. Without this the controller climbs all
// the way to the ceiling against a path it is already overrunning,
// because a path that queues instead of dropping reports no loss at all
// until the buffer finally overflows -- and by then the picture is
// seconds behind.
if loss < LOSS_INCREASE && (!known || self.backlog_ms < QUEUE_CLIMB_MS) {
self.reason = Reason::Climbing;
return self
.target_kbps
.saturating_add(self.limits.ceiling_kbps / CLIMB_FRACTION);
}
self.reason = Reason::Holding;
self.target_kbps
}
fn decide_from_path(&mut self, path: PathView) -> u32 {
match path.estimate_kbps() {
Some(estimate) => {
self.reason = Reason::Fallback;
estimate
}
None => {
// Nothing said and nothing visible. Holding a high target on no
// evidence is how the original failure sustained itself, so this
// decays rather than holds.
self.reason = Reason::Blind;
(self.target_kbps as f32 * BACKOFF) as u32
}
}
}
}
/// Whether a change is big enough to be worth actuating.
///
/// Reaching either end of the band always is: a target pinned to the floor or
/// the ceiling is a fact worth stating even when the step to it was small.
pub fn worth_sending(current: u32, next: u32, limits: Limits) -> bool {
if current == next {
return false;
}
if next == limits.floor_kbps() || next == limits.ceiling_kbps {
return true;
}
let change = current.abs_diff(next) as f32;
change >= current as f32 * DEADBAND
}
#[cfg(test)]
mod tests {
use super::*;
const CEILING: u32 = 10_000;
fn limits() -> Limits {
Limits::new(CEILING)
}
fn controller() -> Controller {
Controller::new(limits())
}
/// A second in which `released` of `total` frames made it, carrying
/// `goodput_kbps` of completed video.
fn report(released: u32, incomplete: u32, goodput_kbps: u64) -> ReceiverReport {
ReceiverReport {
goodput_bps: goodput_kbps * 1000,
released,
incomplete,
never_arrived: 0,
rtt_ms: 180,
}
}
fn healthy() -> ReceiverReport {
report(60, 0, 9_800)
}
/// A path with `backlog` bytes handed over and not yet gone.
fn backlogged(backlog: u64) -> PathView {
PathView {
cwnd_bytes: Some(13_000),
rtt_ms: Some(180),
backlog_bytes: Some(backlog),
}
}
/// The second reported failure, as measured over a 1000-mile link.
///
/// Every frame arrived and every frame completed -- zero loss, zero QUIC
/// congestion events, a flat 180 ms round trip for the whole session -- and
/// the picture was still several seconds behind, because the offer was
/// several times what the path carried and the difference was sitting in
/// the hub's own send buffer. A controller reading loss alone sees a
/// perfect path here and climbs to the ceiling against it.
#[test]
fn a_path_that_queues_instead_of_dropping_must_not_read_as_healthy() {
let mut c = controller();
// Two megabits get through; a megabyte is already queued behind them.
let arriving = report(60, 0, 2_000);
let path = backlogged(1024 * 1024);
for _ in 0..30 {
c.tick(1, Some(arriving), path, false);
}
assert_ne!(
c.reason(),
Reason::Climbing,
"climbing against a path with four seconds of backlog"
);
assert!(
c.target_kbps() < 2_000,
"target {} kbps is at or above what is getting through, so the \
queue can only grow",
c.target_kbps()
);
}
/// The sawtooth: floor, climb to ceiling, collapse, repeat every 20 s.
///
/// Loss alone cannot break this cycle, because on a queueing path loss only
/// appears once the buffer finally overflows -- long after the latency has
/// made the session unplayable, and by then the queue is deep enough that
/// backing off to the floor is the only way out.
#[test]
fn the_target_settles_instead_of_sawtoothing() {
let mut c = controller();
// A steady 2 Mbps path. The backlog is what the last second of
// over-sending left behind, drained at what actually gets through.
let mut backlog: i64 = 0;
let mut seen = Vec::new();
for _ in 0..60 {
let target = c.target_kbps();
// Whatever was asked for above 2 Mbps piles up; the rest drains.
backlog = (backlog + (i64::from(target) - 2_000) * 1000 / 8).clamp(0, 4 * 1024 * 1024);
c.tick(
1,
Some(report(60, 0, 2_000)),
backlogged(backlog as u64),
false,
);
seen.push(c.target_kbps());
}
let settled = &seen[30..];
let (lo, hi) = (
*settled.iter().min().unwrap(),
*settled.iter().max().unwrap(),
);
assert!(
hi - lo <= 1_000,
"target still swinging between {lo} and {hi} kbps after 30 seconds"
);
assert!(
hi <= 2_400,
"settled at {hi} kbps against a path carrying 2000"
);
}
/// The reported depth is what the next session will be judged on, so it
/// has to be right: bytes over the rate they leave at, in milliseconds.
#[test]
fn the_queue_depth_is_reported_as_measured() {
let mut c = controller();
// 250 kB queued behind a 2 Mbps drain is exactly one second.
c.tick(1, Some(report(60, 0, 2_000)), backlogged(250_000), false);
assert_eq!(c.backlog_ms(), 1_000);
// A transport that cannot say must report no queue, not a wrong one.
c.tick(1, Some(report(60, 0, 2_000)), PathView::default(), false);
assert_eq!(c.backlog_ms(), 0);
}
/// The backlog must not become a reason never to climb again.
#[test]
fn a_clear_queue_still_climbs() {
let mut c = controller();
c.set_ceiling(4_000);
let clear = backlogged(0);
for _ in 0..5 {
c.tick(1, Some(report(60, 0, 3_900)), clear, false);
}
assert_eq!(c.reason(), Reason::Climbing);
}
/// The reported failure, as measured: a 10 Mbps offer into a path carrying
/// under three, where *no* frame completed for minutes.
#[test]
fn the_reported_failure_is_corrected_in_seconds() {
let mut c = controller();
assert_eq!(c.target_kbps(), CEILING);
let mut ticks = 0;
while c.target_kbps() > 2_800 && ticks < 10 {
// 46 frames a second starting to arrive, none of them completing.
c.tick(1, Some(report(0, 46, 0)), PathView::default(), false);
ticks += 1;
}
assert!(
c.target_kbps() <= 2_800,
"still at {} kbps after {ticks} seconds",
c.target_kbps(),
);
assert!(
ticks <= 5,
"took {ticks} seconds to stop overrunning the path"
);
}
/// With a goodput measurement to anchor on it should take one step, not
/// several -- that is the whole reason for backing off to what arrived
/// rather than to a fraction of what was asked for.
#[test]
fn a_measured_goodput_is_corrected_in_one_step() {
let mut c = controller();
let sent = c.tick(1, Some(report(12, 48, 2_850)), PathView::default(), false);
assert_eq!(sent, Some(c.target_kbps()));
assert!(
c.target_kbps() < 2_850,
"backed off to {} kbps, which is above what actually arrived",
c.target_kbps(),
);
}
/// The negative that matters most: a path that is fine must be left alone.
#[test]
fn a_healthy_path_is_not_wandered_away_from() {
let mut c = controller();
// The first decision is always stated. The encoder started at whatever
// its environment gave it and this end cannot know that matches, so the
// target is asserted once rather than assumed.
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(CEILING)
);
for _ in 0..60 {
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
None,
"a healthy path produced a bitrate change",
);
}
assert_eq!(c.target_kbps(), CEILING);
}
#[test]
fn the_first_decision_is_always_stated() {
// Otherwise a session whose path happens to match its ceiling never
// tells the encoder anything, and the encoder keeps whatever its
// environment gave it -- which is the failure this replaces.
let mut c = controller();
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(CEILING),
"the opening target was never stated",
);
// And not repeated, now that the encoder has been told.
assert_eq!(c.tick(1, Some(healthy()), PathView::default(), false), None);
}
#[test]
fn a_small_change_is_not_worth_saying() {
// Below the deadband nothing is sent, so the log and the encoder are not
// touched sixty times a minute for changes nobody could see.
let l = limits();
assert!(!worth_sending(5_000, 5_200, l));
assert!(worth_sending(5_000, 5_600, l));
}
#[test]
fn reaching_either_end_of_the_band_is_always_worth_saying() {
let l = limits();
assert!(worth_sending(l.floor_kbps() + 1, l.floor_kbps(), l));
assert!(worth_sending(CEILING - 1, CEILING, l));
}
#[test]
fn the_ceiling_is_never_exceeded() {
let mut c = controller();
for _ in 0..200 {
c.tick(1, Some(healthy()), PathView::default(), false);
assert!(c.target_kbps() <= CEILING, "{} kbps", c.target_kbps());
}
}
#[test]
fn the_floor_is_never_gone_below() {
let mut c = controller();
for _ in 0..200 {
c.tick(1, Some(report(0, 60, 0)), PathView::default(), false);
assert!(
c.target_kbps() >= c.limits().floor_kbps(),
"{} kbps is below the floor",
c.target_kbps(),
);
}
}
#[test]
fn it_climbs_back_after_a_bad_patch() {
let mut c = controller();
for _ in 0..5 {
c.tick(1, Some(report(0, 46, 0)), PathView::default(), false);
}
let bottom = c.target_kbps();
for _ in 0..40 {
c.tick(1, Some(healthy()), PathView::default(), false);
}
assert!(
c.target_kbps() > bottom,
"never recovered from {bottom} kbps"
);
assert_eq!(c.target_kbps(), CEILING, "did not climb all the way back");
}
#[test]
fn a_second_the_hub_starved_is_not_read_as_a_bad_path() {
// While a client resynchronises its deltas are withheld, so fewer frames
// are sent, fewer arrive, goodput reads low and loss reads high. Backing
// off on that would cut the bitrate in response to the hub's own
// decision -- and make every stutter cost bandwidth as well.
let mut c = controller();
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(CEILING)
);
let starved = report(0, 46, 0);
for _ in 0..10 {
assert_eq!(c.tick(1, Some(starved), PathView::default(), true), None);
}
assert_eq!(c.target_kbps(), CEILING, "the hub cut its own bitrate");
assert_eq!(c.reason(), Reason::SelfInflicted);
}
#[test]
fn a_genuinely_bad_second_still_acts_once_the_hub_stops_starving_it() {
// The flag holds, it does not blind. The moment a second is the path's
// own, the same evidence is acted on.
let mut c = controller();
c.tick(1, Some(healthy()), PathView::default(), false);
c.tick(1, Some(report(0, 46, 0)), PathView::default(), true);
assert_eq!(c.target_kbps(), CEILING);
c.tick(1, Some(report(12, 48, 2_850)), PathView::default(), false);
assert!(c.target_kbps() < 2_850, "a real bad second was ignored too");
}
#[test]
fn a_starved_second_does_not_count_towards_silence() {
// It is not a missing report -- one arrived, it just cannot be read.
// Counting it as silence would slide towards the path-based fallback and
// then to decaying blind, which is a different wrong answer.
let mut c = controller();
c.tick(1, Some(healthy()), PathView::default(), false);
for _ in 0..10 {
c.tick(1, Some(report(0, 60, 0)), PathView::default(), true);
}
assert_eq!(c.reason(), Reason::SelfInflicted);
assert_eq!(c.target_kbps(), CEILING);
}
#[test]
fn a_hand_set_bitrate_stands_the_controller_down() {
// How this class of bug gets diagnosed. A controller that overrode a
// person's setting a second later would take the tool away.
let mut c = controller();
c.note_manual_target(1_000);
for _ in 0..30 {
assert_eq!(
c.tick(1, Some(report(0, 60, 0)), PathView::default(), false),
None
);
}
assert_eq!(c.target_kbps(), 1_000);
assert_eq!(c.reason(), Reason::Manual);
}
#[test]
fn constant_quality_has_no_bitrate_to_decide() {
let mut c = controller();
c.set_constant_quality(true);
assert_eq!(
c.tick(1, Some(report(0, 60, 0)), PathView::default(), false),
None
);
assert_eq!(c.reason(), Reason::ConstantQuality);
}
#[test]
fn silence_is_tolerated_briefly_and_then_acted_on() {
let mut c = controller();
// A missed report or two says nothing; the path was fine a second ago.
for _ in 0..(SILENT_TICKS_BEFORE_FALLBACK - 1) {
assert_eq!(c.tick(1, None, PathView::default(), false), None);
assert_eq!(c.target_kbps(), CEILING);
}
// Past that, with nothing visible from this end either, it decays rather
// than holding a high target on no evidence at all -- holding is how the
// original failure sustained itself.
c.tick(1, None, PathView::default(), false);
assert_eq!(c.reason(), Reason::Blind);
assert!(c.target_kbps() < CEILING);
}
#[test]
fn this_ends_view_is_used_only_when_the_far_end_is_silent() {
let mut c = controller();
let path = PathView {
cwnd_bytes: Some(120_000),
rtt_ms: Some(200),
backlog_bytes: None,
};
// A report present means the path view is ignored, however tempting.
c.tick(1, Some(healthy()), path, false);
assert_eq!(c.reason(), Reason::Climbing);
for _ in 0..SILENT_TICKS_BEFORE_FALLBACK {
c.tick(1, None, path, false);
}
assert_eq!(c.reason(), Reason::Fallback);
// 120 KB in flight per 200 ms is 4.8 Mbps.
assert_eq!(c.target_kbps(), 4_800);
}
#[test]
fn nothing_is_decided_while_nobody_is_connected() {
// A box waiting for its first client used to read the silence as a dead
// path and wind itself down to the floor, then jump back up the moment
// somebody arrived. There is no path to have an opinion about yet.
let mut c = controller();
for _ in 0..30 {
assert_eq!(c.tick(0, None, PathView::default(), false), None);
}
assert_eq!(c.target_kbps(), CEILING);
assert_eq!(c.reason(), Reason::NoClients);
}
#[test]
fn a_lowered_ceiling_can_be_raised_again() {
// The trap in folding the box's ceiling together with the one in force:
// after lowering, the only number left to compare against is the lowered
// one, so the client can never get back up.
let mut c = controller();
c.set_ceiling(2_000);
assert_eq!(c.limits().ceiling_kbps, 2_000);
c.set_ceiling(5_000);
assert_eq!(c.limits().ceiling_kbps, 5_000);
assert_eq!(c.box_ceiling_kbps(), CEILING);
}
#[test]
fn a_ceiling_change_is_restated_to_the_encoder() {
// The encoder is the thing that has to act on it, and a target that
// happens to land on the same number is still a different instruction
// when the band around it moved.
let mut c = controller();
assert!(
c.tick(1, Some(healthy()), PathView::default(), false)
.is_some()
);
c.set_ceiling(3_000);
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(3_000)
);
}
#[test]
fn a_client_may_lower_its_ceiling_but_not_raise_it() {
let mut c = controller();
c.set_ceiling(4_000);
assert_eq!(c.limits().ceiling_kbps, 4_000);
assert!(c.target_kbps() <= 4_000, "the target outlived its ceiling");
c.set_ceiling(50_000);
assert_eq!(
c.limits().ceiling_kbps,
CEILING,
"a client raised the ceiling the tier bought",
);
}
#[test]
fn a_ceiling_under_the_floor_does_not_invert_the_band() {
// A misconfigured tier must not produce a floor above its own ceiling,
// which would put every target above the limit it exists to enforce.
let l = Limits::new(500);
assert!(l.floor_kbps() <= l.ceiling_kbps);
assert_eq!(l.clamp(10_000), 500);
}
#[test]
fn a_silent_second_is_not_read_as_a_healthy_one() {
// No frames at all accounted for: nothing sent, or nothing arrived.
// Reading it as no loss would climb straight into a path that may be
// carrying nothing.
let mut c = controller();
let empty = ReceiverReport::default();
assert_eq!(empty.loss(), None);
c.tick(1, Some(empty), PathView::default(), false);
assert_eq!(c.target_kbps(), CEILING, "climbed on an empty report");
}
}
apps/neshub/src/dgram.rs
+204 -31
View File
@@ -143,6 +143,73 @@ impl DatagramSender {
}
}
/// How long deltas may be withheld from a client waiting to resynchronise.
///
/// A bound rather than a belief. Withholding is correct only while the keyframe
/// is actually coming, and if the encoder never produces one -- it refused, it
/// died, the request never reached it -- then withholding forever turns a
/// recoverable freeze into a permanent black screen. Past this the deltas go out
/// again: useless to a desynchronised decoder, but "useless" beats "nothing at
/// all, forever" when the assumption behind the suppression has been proven
/// wrong.
const MAX_RESYNC_WITHHOLD: std::time::Duration = std::time::Duration::from_secs(2);
/// Whether to send delta frames to a client that cannot decode them yet.
///
/// A client that has lost synchronisation asks for a keyframe, and until one
/// arrives every delta frame sent to it is undecodable -- it predicts from
/// pictures that client does not have. Those frames are not merely wasted: quinn
/// writes DATAGRAM frames into a packet before STREAM frames, so a steady stream
/// of deltas takes the space the keyframe needs and starves the one frame that
/// would end the freeze. That is the loop behind "26 keyframe fallbacks, 19 IDR
/// requests": the recovery frame could not get out past the frames that needed
/// it to arrive first.
///
/// So while a client is waiting, its deltas are dropped rather than sent.
#[derive(Debug, Default)]
pub struct ResyncGate {
/// When this client started waiting, or `None` if it is not.
waiting_since: Option<std::time::Instant>,
/// Deltas dropped during the current wait.
withheld: u64,
/// Set once the withhold bound is passed, so it is said once per wait.
gave_up: bool,
}
impl ResyncGate {
/// Whether this delta frame should go out.
///
/// `awaiting` is whether the client has asked for a keyframe and not yet
/// been sent one.
pub fn admit_delta(&mut self, awaiting: bool, now: std::time::Instant) -> bool {
if !awaiting {
self.end_wait();
return true;
}
let since = *self.waiting_since.get_or_insert(now);
if now.duration_since(since) >= MAX_RESYNC_WITHHOLD {
self.gave_up = true;
return true;
}
self.withheld += 1;
false
}
/// A keyframe has gone out, so the wait is over.
pub fn note_keyframe(&mut self) -> Option<(u64, bool)> {
let withheld = self.withheld;
let gave_up = self.gave_up;
self.end_wait();
(withheld > 0).then_some((withheld, gave_up))
}
fn end_wait(&mut self) {
self.waiting_since = None;
self.withheld = 0;
self.gave_up = false;
}
}
/// Frames arriving on `rx` are numbered and sent — deltas as datagrams,
/// keyframes on a reliable stream each when `keyframes_reliable` is set.
///
@@ -161,6 +228,12 @@ pub async fn run_datagram_writer(
mut rx: tokio::sync::mpsc::UnboundedReceiver<Vec<u8>>,
relay_ms: Option<Arc<AtomicU32>>,
keyframes_reliable: bool,
// Set while this client has asked for a keyframe and not yet been sent
// one. Video only; audio has no such notion.
awaiting_keyframe: Option<Arc<std::sync::atomic::AtomicBool>>,
// Counts frames withheld during a resynchronisation, so the bitrate
// controller can tell a second it starved from a second the path did.
withheld: Option<Arc<std::sync::atomic::AtomicU64>>,
) {
let sender = DatagramSender::new(conn.clone(), kind);
let keyframes = keyframes_reliable.then(|| KeyframeSender::new(conn, sender.clone()));
@@ -170,33 +243,54 @@ pub async fn run_datagram_writer(
// debug. Losing datagram support entirely is not, and is worth a warning —
// but only the first time, since it will then be true for every frame.
let mut warned_unsupported = false;
// Where this writer's second went.
//
// The pair that matters is the first two: if frames arrive here already
// 43 ms apart then the hole was made upstream, in the encoder or on the IPC
// hop, and nothing in this file can be the cause. If they arrive evenly and
// leave unevenly, it is made here. A client measured exactly that hole in
// video datagram arrivals while audio — same connection, same congestion
// window, its own writer — stayed at 8 ms.
let mut last_in = std::time::Instant::now();
let mut worst_in_gap = std::time::Duration::ZERO;
let mut worst_send = std::time::Duration::ZERO;
let mut frames: u32 = 0;
let mut last_pace = std::time::Instant::now();
let mut resync = ResyncGate::default();
while let Some(payload) = rx.recv().await {
let t0 = std::time::Instant::now();
worst_in_gap = worst_in_gap.max(t0.duration_since(last_in));
last_in = t0;
frames += 1;
body.clear();
nesprotocol::encode_frame_body(&mut body, MSG_DATA, seq, &payload);
// A keyframe goes on a stream of its own when one will take it. The
// relay timing below is not recorded for it: the write is asynchronous
// by design, so the time this loop spent on it says nothing.
// A client that cannot decode has asked for a keyframe; until it gets
// one, everything else sent to it is undecodable and takes the space the
// keyframe needs. See `ResyncGate`.
if let Some(ref awaiting) = awaiting_keyframe {
if nesprotocol::reliable::video_is_keyframe(&payload) {
awaiting.store(false, Ordering::Relaxed);
if let Some((withheld, gave_up)) = resync.note_keyframe() {
debug!(
"{label}: resynchronised after withholding {withheld} frame(s){}",
if gave_up {
", having given up waiting"
} else {
""
}
);
}
} else if !resync
.admit_delta(awaiting.load(Ordering::Relaxed), std::time::Instant::now())
{
// The sequence number deliberately does *not* advance. A
// withheld frame was never sent, so leaving a hole would make
// the receiver count it as lost -- and that count is what the
// bitrate controller reads. The hub would then lower the
// bitrate because of frames it chose not to send, which is a
// controller reacting to its own decision rather than to the
// path. Reusing the number is safe precisely because nothing
// went out under it.
//
// Saying so is the other half: the receiver still reports fewer
// frames this second, and without this count the controller has
// no way to tell that second from one the path ruined.
if let Some(ref withheld) = withheld {
withheld.fetch_add(1, Ordering::Relaxed);
}
continue;
}
}
if let Some(ref keyframes) = keyframes
&& video_wants_reliable(&payload)
{
@@ -229,19 +323,6 @@ pub async fn run_datagram_writer(
}
Err(e) => debug!("{label}: dropping frame {seq}: {e}"),
}
worst_send = worst_send.max(t0.elapsed());
if last_pace.elapsed() >= std::time::Duration::from_secs(1) {
last_pace = std::time::Instant::now();
debug!(
"{label}: {frames} frames, worst gap between frames in {:.1}ms, worst send {:.1}ms",
worst_in_gap.as_secs_f64() * 1000.0,
worst_send.as_secs_f64() * 1000.0,
);
worst_in_gap = std::time::Duration::ZERO;
worst_send = std::time::Duration::ZERO;
frames = 0;
}
seq = seq.wrapping_add(1);
}
@@ -252,3 +333,95 @@ pub async fn run_datagram_writer(
}
debug!("{label} datagram writer exiting (channel closed)");
}
#[cfg(test)]
mod resync_gate_tests {
use super::{MAX_RESYNC_WITHHOLD, ResyncGate};
use std::time::{Duration, Instant};
#[test]
fn a_client_that_can_decode_gets_everything() {
// The negative that matters: nothing is withheld from a healthy client.
let mut gate = ResyncGate::default();
let now = Instant::now();
for i in 0..1000 {
assert!(gate.admit_delta(false, now + Duration::from_millis(i)));
}
}
#[test]
fn a_waiting_client_is_not_sent_frames_it_cannot_decode() {
let mut gate = ResyncGate::default();
let now = Instant::now();
for i in 0..60 {
assert!(
!gate.admit_delta(true, now + Duration::from_millis(i * 16)),
"frame {i} went to a client with no reference to decode it against",
);
}
}
#[test]
fn the_keyframe_ends_the_wait_and_reports_the_cost() {
let mut gate = ResyncGate::default();
let now = Instant::now();
for i in 0..5 {
gate.admit_delta(true, now + Duration::from_millis(i * 16));
}
assert_eq!(gate.note_keyframe(), Some((5, false)));
// And the next delta goes out immediately.
assert!(gate.admit_delta(false, now + Duration::from_millis(100)));
}
#[test]
fn a_keyframe_with_nothing_withheld_says_nothing() {
// So an ordinary periodic keyframe does not log a recovery that did not
// happen.
let mut gate = ResyncGate::default();
assert_eq!(gate.note_keyframe(), None);
}
#[test]
fn withholding_gives_up_rather_than_going_dark_forever() {
// The bound. Withholding is only correct while the keyframe is actually
// coming; if the encoder never produces one, suppressing forever turns a
// recoverable freeze into a permanent black screen. Undecodable frames
// beat no frames once the assumption is disproven.
let mut gate = ResyncGate::default();
let now = Instant::now();
assert!(!gate.admit_delta(true, now));
assert!(!gate.admit_delta(true, now + MAX_RESYNC_WITHHOLD - Duration::from_millis(1)));
assert!(
gate.admit_delta(true, now + MAX_RESYNC_WITHHOLD),
"still withholding after the keyframe plainly is not coming",
);
}
#[test]
fn giving_up_is_reported_when_the_keyframe_finally_lands() {
let mut gate = ResyncGate::default();
let now = Instant::now();
gate.admit_delta(true, now);
gate.admit_delta(true, now + MAX_RESYNC_WITHHOLD);
let (withheld, gave_up) = gate.note_keyframe().expect("something was withheld");
assert_eq!(withheld, 1);
assert!(gave_up, "the wait timed out and nothing said so");
}
#[test]
fn a_second_wait_starts_fresh() {
// Otherwise the first wait's elapsed time carries over and the second is
// abandoned immediately, or its count reports the wrong recovery.
let mut gate = ResyncGate::default();
let now = Instant::now();
gate.admit_delta(true, now);
gate.note_keyframe();
let later = now + Duration::from_secs(60);
assert!(
!gate.admit_delta(true, later),
"the new wait was not honoured"
);
assert_eq!(gate.note_keyframe(), Some((1, false)));
}
}
apps/neshub/src/main.rs
+155 -12
View File
@@ -1,3 +1,4 @@
mod control;
mod dgram;
mod ipc_listener;
mod keyframe;
@@ -14,7 +15,7 @@ use iroh::endpoint::presets;
use crate::session::SessionManager;
use crate::ticket::NestriTicket;
use nesprotocol::ALPN;
use nesprotocol::{ALPNS, Carrier};
#[derive(Parser, Debug)]
#[command(name = "neshub")]
@@ -73,6 +74,16 @@ struct Args {
#[arg(long, env = "NESTRI_AUDIO_BITRATE", default_value_t = 64)]
audio_bitrate_per_channel: u32,
/// Ceiling on the video bitrate, in kbps.
///
/// Set by `nesinit` from the boot descriptor's video limits, which come
/// from the tier the box was sized for. Absent means nobody said -- which is
/// not a licence to send whatever the encoder defaults to, since that is
/// precisely how every session came to offer 10 Mbps regardless of what the
/// path could carry. Unset is reported, and a conservative ceiling is used.
#[arg(long, env = "NESTRI_MAX_BITRATE")]
max_bitrate_kbps: Option<u32>,
/// Socket nescope sends screenshots on. neshub listens; nescope dials out.
#[arg(
long,
@@ -82,6 +93,14 @@ struct Args {
screenshot_ipc: PathBuf,
}
/// What to assume when nobody said.
///
/// Deliberately modest. A ceiling that was never set should not behave like an
/// unlimited one: the whole failure this exists to fix was a session offering
/// 10 Mbps into a path carrying under three, because no number had ever been
/// chosen and the encoder's own default stood in for one.
const DEFAULT_MAX_BITRATE_KBPS: u32 = 4_000;
#[tokio::main]
async fn main() -> Result<()> {
tracing_subscriber::fmt()
@@ -93,7 +112,7 @@ async fn main() -> Result<()> {
let args = Args::parse();
let mut builder = iroh::Endpoint::builder(presets::N0)
.alpns(vec![ALPN.to_vec()])
.alpns(ALPNS.iter().map(|a| a.to_vec()).collect::<Vec<_>>())
.transport_config(crate::dgram::media_transport_config());
match args.relay.as_str() {
@@ -117,6 +136,15 @@ async fn main() -> Result<()> {
}
}
match args.max_bitrate_kbps {
Some(kbps) => tracing::info!("video ceiling: {kbps} kbps, from the boot descriptor"),
None => tracing::warn!(
"no video ceiling on the boot descriptor; using {DEFAULT_MAX_BITRATE_KBPS} kbps. \
A box sized by a tier is told its ceiling -- if this is one, the descriptor did \
not carry it."
),
}
let endpoint = builder.bind().await?;
let endpoint_addr = endpoint.addr();
let ep_id = endpoint_addr.id;
@@ -134,6 +162,14 @@ async fn main() -> Result<()> {
let session_manager = Arc::new(SessionManager::new());
// One controller for the box, not one per client: there is one encoder, so
// there is one bitrate, and the client having the worst time is the one it
// has to answer.
let box_ceiling_kbps = args.max_bitrate_kbps.unwrap_or(DEFAULT_MAX_BITRATE_KBPS);
let controller = Arc::new(tokio::sync::Mutex::new(control::Controller::new(
control::Limits::new(box_ceiling_kbps),
)));
// IDR / encode settings command channel: input reader → nescapture
let (cmd_tx, mut cmd_rx) = tokio::sync::mpsc::unbounded_channel::<Vec<u8>>();
{
@@ -181,23 +217,105 @@ async fn main() -> Result<()> {
args.audio_channels,
args.audio_bitrate_per_channel
);
let controller = controller.clone();
let cmd_tx = cmd_tx.clone();
tokio::spawn(async move {
// Ten times a second, and only when asked for. Pairs with
// nescapture's rate probe: that measures how fast the encoder
// follows a new bitrate, this shows how fast the queue downstream
// of it responds, and the slower of the two is the fastest a
// control loop can usefully run. At one sample a second neither is
// visible -- a queue that fills and drains inside a second looks
// like a queue that was never there.
if std::env::var("NESHUB_BACKLOG_TRACE").is_ok_and(|v| v != "0" && !v.is_empty()) {
let mgr = mgr.clone();
let controller = controller.clone();
tokio::spawn(async move {
let mut fast = tokio::time::interval(std::time::Duration::from_millis(100));
tracing::info!("backlog trace on, 10 Hz");
loop {
fast.tick().await;
let backlogs = mgr.backlog_bytes().await;
if backlogs.is_empty() {
continue;
}
let target = controller.lock().await.target_kbps();
for bytes in backlogs {
// Against the target rather than a measured drain:
// this is a trace to read afterwards, and a number
// divided by a second measurement is two things
// moving at once.
let ms = bytes.saturating_mul(8) / u64::from(target.max(1));
tracing::info!("backlog {ms} ms ({bytes} bytes) at {target} kbps");
}
}
});
}
let mut interval = tokio::time::interval(std::time::Duration::from_secs(1));
loop {
interval.tick().await;
// One decision a second, on the same tick as the stats, because
// a report describes the second that just passed and there is
// nothing to gain from deciding more often than they arrive.
{
let clients = mgr.client_count().await;
let (report, path, self_inflicted) = mgr.worst_report().await;
let mut controller = controller.lock().await;
if let Some(kbps) = controller.tick(clients, report, path, self_inflicted) {
let mut cmd = vec![nesprotocol::MSG_ENCODE_SETTINGS];
nesprotocol::encode_bitrate_only(&mut cmd, kbps);
if cmd_tx.send(cmd).is_err() {
tracing::warn!("encoder command channel closed");
} else {
// Trace, not info: under a path that keeps moving
// this fires every second, and a per-second line at
// info buries everything worth reading.
tracing::trace!(
"video ceiling {}/{} kbps: {:?}",
kbps,
controller.limits().ceiling_kbps,
controller.reason(),
);
}
}
}
let clients = mgr.client_count().await as u8;
let bitrate = mgr.video_bitrate_bps();
let (key_bps, delta_bps, keyframes) = mgr.video_breakdown();
let (pipeline_p50_ms, pipeline_p95_ms, pipeline_max_ms) = mgr.pipeline_delays();
let audio_kbps = mgr.audio_bitrate_kbps();
let relay_ms = mgr.relay_ms();
let mut buf = Vec::with_capacity(15);
let mut buf = Vec::with_capacity(34);
nesprotocol::stats::encode_hub_stats(
&mut buf,
clients,
bitrate,
key_bps.saturating_add(delta_bps),
relay_ms,
audio_kbps,
audio_channels,
);
{
let controller = controller.lock().await;
nesprotocol::stats::encode_video_breakdown(
&mut buf,
&nesprotocol::stats::VideoBreakdown {
key_bps,
delta_bps,
keyframes,
target_kbps: controller.target_kbps(),
ceiling_kbps: controller.limits().ceiling_kbps,
reason: controller.reason() as u8,
manual: u8::from(controller.mode() == nesprotocol::ControlMode::Manual),
box_ceiling_kbps: controller.box_ceiling_kbps(),
pipeline_p50_ms,
pipeline_p95_ms,
pipeline_max_ms,
backlog_ms: controller.backlog_ms().min(u32::from(u16::MAX)) as u16,
},
);
}
mgr.broadcast_stats(buf).await;
}
});
@@ -262,18 +380,43 @@ async fn main() -> Result<()> {
match incoming.await {
Ok(conn) => {
let remote_id = conn.remote_id();
tracing::info!(remote = %remote_id.fmt_short(), "client connected");
let session = session::ClientSession::new(
// A client opens one connection per kind of traffic, so the
// ALPN says which this is and the endpoint id says whose.
let Some(carrier) = Carrier::from_alpn(conn.alpn()) else {
tracing::warn!(
remote = %remote_id.fmt_short(),
"connection with an unknown ALPN; closing"
);
conn.close(0u32.into(), b"unknown alpn");
continue;
};
tracing::info!(
remote = %remote_id.fmt_short(),
carrier = carrier.label(),
"client connected"
);
mgr.attach(
remote_id,
carrier,
conn.clone(),
input_broadcast_tx.clone(),
session_manager.relay_ms_atomic(),
cmd_tx.clone(),
);
mgr.add_session(remote_id, session).await;
controller.clone(),
)
.await;
let mgr_clone = mgr.clone();
let conn_clone = conn.clone();
tokio::spawn(async move {
conn_clone.closed().await;
conn.closed().await;
// Any one of them going means the session goes. A client
// left holding audio and input but no video is not a
// degraded session, it is a stuck one, and a clean
// reconnect is both simpler to reason about and quicker
// than whatever partial recovery would be built here.
tracing::debug!(
remote = %remote_id.fmt_short(),
carrier = carrier.label(),
"carrier closed, ending session"
);
mgr_clone.remove_session(&remote_id).await;
});
}
apps/neshub/src/session.rs
+773 -171
View File
File diff suppressed because it is too large Load Diff
apps/nesinit/src/lib.rs
+1
View File
@@ -14,6 +14,7 @@
pub mod filesystems;
pub mod payload;
pub mod platform;
pub mod reap;
pub mod services;
pub mod session;
apps/nesinit/src/main.rs
+2
View File
@@ -45,6 +45,7 @@ fn main() -> anyhow::Result<()> {
// nothing else in this guest is an init system, so until this runs there
// is no `/proc` to score this process in and nowhere to put a socket.
nesinit::filesystems::establish();
nesinit::platform::describe();
// Everything a distribution's init scripts used to do, and nothing else is
// going to: a hostname, the box's address, the directories a session's
@@ -80,6 +81,7 @@ fn main() -> anyhow::Result<()> {
Ok(outcome) => tracing::info!(?outcome, "the session ended"),
Err(error) => tracing::error!(%error, "the session failed"),
}
nesinit::platform::report_steal();
// Before anything below waits on a pid: the reaper runs on this runtime's
// threads, and two things calling `wait` is what the registry exists to
apps/nesinit/src/platform.rs
+148
View File
@@ -0,0 +1,148 @@
// What the kernel under this box actually chose, said once, at debug.
//
// Several of the settings that decide a box's latency are not decided by the
// image. The clocksource is picked at boot and can be demoted by a watchdog,
// the idle driver loads only when the host or the command line asks for it,
// and the preemption model is a boot parameter. None of them is visible from
// outside, and a box has no shell to ask with, so this reads them from sysfs
// and procfs and logs them. Enable with `RUST_LOG=nesinit=debug` on the kernel
// command line.
use std::path::Path;
/// Clocksources a vDSO can read without entering the kernel.
///
/// Anything else makes every `clock_gettime` a syscall, and in a guest an
/// emulated one: `hpet` is an MMIO read the host has to trap. A Windows game
/// polls its performance counter constantly, so that is a cost paid thousands
/// of times a frame, and nothing reports it.
const VDSO_CLOCKSOURCES: &[&str] = &["tsc", "kvm-clock"];
/// Log the kernel's timing and idle choices. Needs `/proc` and `/sys`.
pub fn describe() {
let clocksource = read("/sys/devices/system/clocksource/clocksource0/current_clocksource");
let cmdline = read("/proc/cmdline").unwrap_or_default();
tracing::debug!(
clocksource = clocksource.as_deref().unwrap_or("unknown"),
available = read("/sys/devices/system/clocksource/clocksource0/available_clocksource")
.as_deref()
.unwrap_or("unknown"),
idle_driver = read("/sys/devices/system/cpu/cpuidle/current_driver")
.as_deref()
.unwrap_or("none"),
idle_governor = read("/sys/devices/system/cpu/cpuidle/current_governor_ro")
.as_deref()
.unwrap_or("none"),
// Present only while the haltpoll governor is built in, and only
// meaningful while it is the governor in use.
halt_poll_ns = read("/sys/module/haltpoll/parameters/guest_halt_poll_ns")
.as_deref()
.unwrap_or("n/a"),
// The build's default when absent. The mode actually in effect is only
// readable through debugfs, which this kernel does not have.
preempt = kernel_parameter(&cmdline, "preempt").unwrap_or("default"),
cpus = std::thread::available_parallelism().map_or(0, usize::from),
kernel = read("/proc/sys/kernel/version")
.as_deref()
.unwrap_or("unknown"),
"platform"
);
if let Some(source) = clocksource.as_deref()
&& !VDSO_CLOCKSOURCES.contains(&source)
{
tracing::warn!(
clocksource = source,
"every clock read in this box is a syscall; the kernel did not trust a faster clock"
);
}
}
/// Log how much of this box's CPU time the host took back. Call once, at the
/// end of a session: the counters are cumulative since boot.
pub fn report_steal() {
let Some(stat) = read("/proc/stat") else {
return;
};
let Some((steal, total)) = steal_of(&stat) else {
return;
};
// Parts per thousand, so the log carries an integer and no float
// formatting decides how small a number reads as zero.
let permille = (steal * 1000).checked_div(total).unwrap_or(0);
tracing::debug!(
steal_ticks = steal,
total_ticks = total,
permille,
"cpu time stolen by the host"
);
}
/// Steal and total ticks from the aggregate `cpu` line of `/proc/stat`.
///
/// Columns are user, nice, system, idle, iowait, irq, softirq, steal, and then
/// guest time, which the kernel already counts inside user and nice. Summing
/// past steal would count it twice.
fn steal_of(stat: &str) -> Option<(u64, u64)> {
let line = stat.lines().find(|l| l.starts_with("cpu "))?;
let fields: Vec<u64> = line
.split_whitespace()
.skip(1)
.take(8)
.map(str::parse)
.collect::<Result<_, _>>()
.ok()?;
let steal = *fields.get(7)?;
Some((steal, fields.iter().sum()))
}
/// The value of one bare `name=value` kernel parameter.
fn kernel_parameter<'a>(cmdline: &'a str, name: &str) -> Option<&'a str> {
cmdline
.split_whitespace()
.find_map(|word| word.strip_prefix(name)?.strip_prefix('='))
.filter(|value| !value.is_empty())
}
fn read(path: impl AsRef<Path>) -> Option<String> {
std::fs::read_to_string(path)
.ok()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_parameter_is_found_by_its_whole_name() {
let cmdline = "cpuidle_haltpoll.force=1 console=hvc0 preempt=full ro";
assert_eq!(kernel_parameter(cmdline, "preempt"), Some("full"));
assert_eq!(
kernel_parameter(cmdline, "cpuidle_haltpoll.force"),
Some("1")
);
}
/// `preempt` must not match `preempt_foo=`, which is a different parameter.
#[test]
fn a_longer_name_sharing_the_prefix_is_not_a_match() {
assert_eq!(kernel_parameter("preempt_foo=x", "preempt"), None);
assert_eq!(kernel_parameter("console=hvc0", "preempt"), None);
}
#[test]
fn steal_is_the_eighth_column_and_guest_time_is_not_counted_twice() {
// user nice system idle iowait irq softirq steal guest guest_nice
let stat = "cpu 100 0 50 800 10 5 5 30 999 999\ncpu0 1 2 3 4 5 6 7 8 9 10\n";
assert_eq!(steal_of(stat), Some((30, 1000)));
}
#[test]
fn a_stat_without_a_cpu_line_reports_nothing() {
assert_eq!(steal_of("intr 1 2 3\n"), None);
assert_eq!(steal_of("cpu 1 2 3\n"), None);
}
}
apps/nesinit/src/services.rs
+99 -10
View File
@@ -41,6 +41,7 @@ use tokio::sync::mpsc::{Receiver, Sender};
use crate::reap::{Waiters, Watched};
use crate::workload::Failure;
use nesprotocol::lifecycle::VideoLimits;
/// A service that died, and how.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -60,7 +61,13 @@ pub trait Services {
/// Called once, after the shares are mounted and before anything may be
/// launched. An empty stack is legitimate: a box with no services still
/// boots, and a caller can still launch something that needs none.
fn bring_up(&mut self) -> Result<Vec<String>, Failure>;
///
/// `video` comes from the descriptor and reaches the services that read it.
/// It has to arrive here rather than later because a service configured
/// after it is already running has a window in which it is not configured,
/// and for a bitrate ceiling that window is a session streaming at whatever
/// default it started with.
fn bring_up(&mut self, video: VideoLimits) -> Result<Vec<String>, Failure>;
/// Deaths, as they happen.
///
@@ -160,6 +167,17 @@ pub const SERVICE_UID: u32 = 1000;
/// here: the directory belongs to the service user and is not writable by the
/// workload, which is the property [`crate::ticket::Untrusted`] depends on.
pub const AUDIO_DIR: &str = "/run/pipewire";
/// Where the PulseAudio protocol is served, in [`AUDIO_DIR`] for the reason
/// audio's own socket is.
///
/// pipewire-pulse is told this path by its configuration file in the image,
/// which is not something this program can pass it, so the two are compared by
/// a test rather than trusted to agree.
pub const PULSE_SOCKET: &str = "/run/pipewire/pulse-native";
/// [`PULSE_SOCKET`] as a PulseAudio client is told it.
pub const PULSE_SERVER: &str = "unix:/run/pipewire/pulse-native";
pub const SERVICE_GID: u32 = 1000;
/// Where a service's runtime sockets live.
@@ -273,6 +291,26 @@ pub const STACK: &[Service] = &[
umask: None,
ready: None,
},
Service {
name: "pipewire-pulse",
argv: &["/usr/bin/pipewire-pulse"],
env: &[
("XDG_RUNTIME_DIR", RUNTIME_DIR),
("PIPEWIRE_RUNTIME_DIR", AUDIO_DIR),
("DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"),
],
user: Some((SERVICE_UID, SERVICE_GID)),
// Optional for the reason the sender is: everything that speaks
// PipeWire itself is unaffected, and a session with sound missing is
// degraded rather than unusable.
cost: "anything that only speaks PulseAudio plays silently, and Wine is one",
required: false,
// The workload is its client, and is not this user.
umask: Some(0),
// Nothing in this table connects to it, but the workload does, and the
// workload is started after the table.
ready: Some(PULSE_SOCKET),
},
Service {
name: "neswire",
argv: &["/usr/bin/neswire"],
@@ -308,6 +346,13 @@ pub struct Stack {
running: Vec<(&'static str, Watched)>,
deaths: Receiver<Died>,
reported: Sender<Died>,
/// What the host said this box may spend on video, from the descriptor.
///
/// Held here because `spawn` is where it reaches a service, and `spawn`
/// takes a `&'static Service` whose `env` is a fixed table -- a value that
/// arrives at runtime has no route through it otherwise. The same problem
/// `RUST_LOG` has, solved the same way.
video: VideoLimits,
}
impl Stack {
@@ -328,6 +373,7 @@ impl Stack {
running: Vec::new(),
deaths,
reported,
video: VideoLimits::default(),
}
}
@@ -390,6 +436,15 @@ impl Stack {
if let Ok(filter) = std::env::var("RUST_LOG") {
command.env("RUST_LOG", filter);
}
// The descriptor's video limits, for the services that read them. Same
// shape of problem as `RUST_LOG` above -- `env_clear` drops everything
// and the service table is a fixed list of literals, so a value that
// only exists at runtime has no other route in. `neshub` reads this
// through the clap `env =` attribute it already uses for every other
// setting.
if let Some(kbps) = self.video.bitrate_kbps {
command.env("NESTRI_MAX_BITRATE", kbps.to_string());
}
// The service's own entry last, so a service that states one of these
// for itself wins over the defaults above.
command.envs(service.env.iter().copied());
@@ -451,7 +506,8 @@ impl Stack {
}
impl Services for Stack {
fn bring_up(&mut self) -> Result<Vec<String>, Failure> {
fn bring_up(&mut self, video: VideoLimits) -> Result<Vec<String>, Failure> {
self.video = video;
let mut up = Vec::new();
// Lifted out so the loop does not hold a borrow of `self` across the
// start it is asking for.
@@ -604,6 +660,9 @@ pub mod double {
/// only thing under test.
pub struct Double {
pub brought_up: usize,
/// What the last `bring_up` was told, so a test can assert the limits
/// reached the stack rather than assuming they did.
pub video: VideoLimits,
pub failure: Option<Failure>,
pub names: Vec<String>,
deaths: Receiver<Died>,
@@ -625,6 +684,7 @@ pub mod double {
names: vec!["dbus-system".into(), "neshub".into()],
deaths,
report,
video: VideoLimits::default(),
}
}
@@ -637,8 +697,9 @@ pub mod double {
}
impl Services for Double {
fn bring_up(&mut self) -> Result<Vec<String>, Failure> {
fn bring_up(&mut self, video: VideoLimits) -> Result<Vec<String>, Failure> {
self.brought_up += 1;
self.video = video;
match &self.failure {
Some(failure) => Err(failure.clone()),
None => Ok(self.names.clone()),
@@ -853,14 +914,42 @@ mod tests {
/// owner -- and the workload is not the owner.
#[test]
fn the_audio_socket_is_reachable_by_a_user_who_does_not_own_it() {
let pipewire = STACK
for name in ["pipewire", "pipewire-pulse"] {
let service = STACK
.iter()
.find(|s| s.name == name)
.unwrap_or_else(|| panic!("{name} is in the table"));
assert_eq!(
service.umask,
Some(0),
"with any other umask the game finds {name}'s socket and cannot open it"
);
}
}
/// The PulseAudio socket's path is written three times: here, in the
/// client address the workload is given, and in pipewire-pulse's own
/// configuration in the image. If any of them moves on its own, the game
/// finds no server and plays silently, and nothing fails.
#[test]
fn pulse_is_served_where_the_workload_is_told_to_look() {
assert!(
PULSE_SOCKET.starts_with(AUDIO_DIR),
"the workload can only reach sockets in {AUDIO_DIR}"
);
assert_eq!(PULSE_SERVER, format!("unix:{PULSE_SOCKET}"));
let pulse = STACK
.iter()
.find(|s| s.name == "pipewire")
.expect("audio is in the table");
assert_eq!(
pipewire.umask,
Some(0),
"with any other umask the game finds the socket and cannot open it"
.find(|s| s.name == "pipewire-pulse")
.expect("pulse is in the table");
assert_eq!(pulse.ready, Some(PULSE_SOCKET));
let config =
include_str!("../../../build/etc/pipewire/pipewire-pulse.conf.d/50-nestri.conf");
assert!(
config.contains(&format!("\"{PULSE_SERVER}\"")),
"pipewire-pulse is configured to listen somewhere other than {PULSE_SOCKET}"
);
}
apps/nesinit/src/session.rs
+56 -8
View File
@@ -266,19 +266,19 @@ where
}
booted = true;
// The drives, then the shares, then the services.
// The overlays, then the shares, then the services.
//
// **Drives first, and one `Mounted` between them.** A drive is
// a filesystem this end mounts itself, so a share whose target
// lives under one has to find it already there. The host is
// **Overlays first, and one `Mounted` between them.** An
// overlay is a filesystem this end mounts itself, so a share
// whose target lives under one has to find it already there. The host is
// told once, after both, because `Mounted` answers "is the
// content where the descriptor said" and there is one answer to
// that -- sending it twice made the host read the second as a
// reply to something it had not asked.
if let Err(failure) = workload.mount_drives(&descriptor.drives) {
if let Err(failure) = workload.mount_overlays(&descriptor.overlays) {
// Said before it is returned. `Refused` ends the session
// either way; without the message the host sees a box that
// stopped and has to guess between a drive, a share and a
// stopped and has to guess between an overlay, a share and a
// service -- which is the whole reason these are reported
// separately.
send(
@@ -311,7 +311,7 @@ where
// A box whose own services will not come up cannot be launched
// into, so this is refused rather than reported and carried on
// from — unlike a launch, which is the caller's to correct.
match services.bring_up() {
match services.bring_up(descriptor.video) {
Ok(up) => {
tracing::info!(services = up.len(), "the box is ready to be launched into");
send(&mut writer, &GuestToHost::Initialized { services: up }).await?
@@ -516,7 +516,8 @@ mod tests {
at: "/mnt/user".into(),
ro: false,
}],
drives: Vec::new(),
overlays: Vec::new(),
video: Default::default(),
}
}
@@ -715,6 +716,53 @@ mod tests {
);
}
#[tokio::test]
async fn the_descriptors_video_limits_reach_the_services() {
// The ceiling is useless if it stops at the descriptor. `neshub` is the
// only thing that can enforce it and it is a service, so the number has
// to survive the whole way from the boot document to the spawn.
let (guest, host) = tokio::io::duplex(4096);
let mut caller = Caller::new(host);
let session = spawn(guest, Given::new(Double::exits_when_stopped(Exit::code(0))));
let mut given = descriptor();
given.video.bitrate_kbps = Some(8_000);
caller.expect_ready().await;
caller
.say(&HostToGuest::Boot {
descriptor: Box::new(given),
})
.await;
caller.expect_booted().await;
caller.say(&HostToGuest::Shutdown).await;
let (_, _, services) = session.await.unwrap();
assert_eq!(services.video.bitrate_kbps, Some(8_000));
}
#[tokio::test]
async fn a_box_told_nothing_about_video_says_so_rather_than_inventing_a_limit() {
// "Unsaid" must not arrive as a number. A stack that cannot tell the two
// apart cannot log that it was never told, and a ceiling nobody set is
// exactly how every session came to offer 10 Mbps.
let (guest, host) = tokio::io::duplex(4096);
let mut caller = Caller::new(host);
let session = spawn(guest, Given::new(Double::exits_when_stopped(Exit::code(0))));
caller.expect_ready().await;
caller
.say(&HostToGuest::Boot {
descriptor: Box::new(descriptor()),
})
.await;
caller.expect_booted().await;
caller.say(&HostToGuest::Shutdown).await;
let (_, _, services) = session.await.unwrap();
assert_eq!(services.video.bitrate_kbps, None);
}
#[tokio::test]
async fn a_launch_runs_what_it_names_and_is_reported_by_its_id() {
let (guest, host) = tokio::io::duplex(4096);
apps/nesinit/src/workload.rs
+271 -81
View File
@@ -11,7 +11,7 @@ use std::future::Future;
use std::io;
use std::pin::Pin;
use nesprotocol::lifecycle::{Drive, Exec, Exit, Mount};
use nesprotocol::lifecycle::{Exec, Exit, Mount, Overlay};
use std::os::unix::process::CommandExt;
@@ -41,8 +41,9 @@ pub trait Workload {
/// Make the shares the descriptor names, where it says to put them.
fn mount(&mut self, mounts: &[Mount]) -> Result<(), Failure>;
/// Mount drives
fn mount_drives(&mut self, drives: &[Drive]) -> Result<(), Failure>;
/// Stack each overlay the descriptor names: its build image, its writable
/// layer, and the two together where it says.
fn mount_overlays(&mut self, overlays: &[Overlay]) -> Result<(), Failure>;
/// Start the command the descriptor names.
///
@@ -132,9 +133,9 @@ impl Workload for Process {
Ok(())
}
fn mount_drives(&mut self, drives: &[Drive]) -> Result<(), Failure> {
for drive in drives {
mount_drive(drive)?;
fn mount_overlays(&mut self, overlays: &[Overlay]) -> Result<(), Failure> {
for overlay in overlays {
mount_overlay(overlay)?;
}
Ok(())
}
@@ -302,6 +303,9 @@ const GRAPHICS: &[(&str, &str)] = &[
// reach and both are told where. Without this a game renders and plays
// silently, having looked under its own uid and found nothing.
("PIPEWIRE_RUNTIME_DIR", crate::services::AUDIO_DIR),
// The same, for a client that speaks PulseAudio instead. It does not read
// the variable above, and its default is under its own runtime directory.
("PULSE_SERVER", crate::services::PULSE_SERVER),
];
/// Mount one share where the descriptor says to put it.
@@ -340,35 +344,104 @@ fn mount_share(share: &Mount) -> Result<(), Failure> {
/// filesystem this mounts. A descriptor cannot name another.
const FSTYPE: &std::ffi::CStr = c"virtiofs";
/// Mounts block device instead of virtiofs share
fn mount_drive(drive: &Drive) -> Result<(), Failure> {
// Checked before anything is created: a descriptor this component cannot
// act on should leave no directory behind to confuse whoever reads the
// failure.
let (source, target, flags) = options_drive(drive)?;
/// Stack one overlay: the build image, the box's writable layer, and the two
/// together at `at`.
///
/// Each step's failure names the step, because "the install did not mount"
/// has three different causes and each one is fixed somewhere else: a build
/// image the kernel cannot read, an upper layer that was never formatted, and
/// an overlay the kernel refused.
fn mount_overlay(overlay: &Overlay) -> Result<(), Failure> {
// Everything that can be refused without touching the filesystem is
// refused first, so a descriptor this cannot act on leaves nothing behind.
let plan = OverlayPlan::new(overlay)?;
// The mount point may not exist yet: a share can land anywhere the
// descriptor names, including a directory no image created.
std::fs::create_dir_all(&drive.at).map_err(|error| failed_drive(drive, error))?;
mount_one(
&plan.lower,
&plan.lower_at,
c"erofs",
plan.lower_flags,
None,
)
.map_err(|error| plan.failed("the build image", &plan.lower_at, error))?;
mount_one(&plan.upper, &plan.rw_at, c"ext4", plan.upper_flags, None)
.map_err(|error| plan.failed("the writable layer", &plan.rw_at, error))?;
// SAFETY: mount takes two paths, a filesystem name and a flag word, all
// of which outlive the call, and no options string.
for dir in [&plan.upper_dir, &plan.work_dir] {
std::fs::create_dir_all(as_path(dir))
.map_err(|error| plan.failed("the writable layer", dir, error))?;
}
// **The upper directory takes the build's root ownership.** overlayfs
// shows a merged directory with the attributes of its upper half when it
// has one, and this one always does -- so an upper directory this init
// created, `root:root 0755`, would make the install's top directory
// unwritable to the workload, whatever the build image says. Copied from
// the lower root rather than named here: which uid the workload runs as is
// the host's decision, and the host already made it when it packed the
// image.
let (uid, gid, mode) = ownership(&plan.lower_at)
.map_err(|error| plan.failed("the build image", &plan.lower_at, error))?;
set_ownership(&plan.upper_dir, uid, gid, mode)
.map_err(|error| plan.failed("the writable layer", &plan.upper_dir, error))?;
mount_one(
c"overlay",
&plan.at,
c"overlay",
plan.overlay_flags,
Some(&plan.overlay_data),
)
.map_err(|error| plan.failed("the overlay", &plan.at, error))?;
Ok(())
}
/// Mount one filesystem, creating its mount point first.
fn mount_one(
source: &std::ffi::CStr,
target: &std::ffi::CStr,
fstype: &std::ffi::CStr,
flags: libc::c_ulong,
data: Option<&std::ffi::CStr>,
) -> io::Result<()> {
// The mount point may not exist yet: the descriptor can name anywhere,
// including a directory no image created.
std::fs::create_dir_all(as_path(target))?;
// SAFETY: every pointer is to a nul-terminated string that outlives the
// call, and a null data pointer is what mount(2) takes for "no options".
let mounted = unsafe {
libc::mount(
source.as_ptr(),
target.as_ptr(),
FSTYPE_DRIVE.as_ptr(),
fstype.as_ptr(),
flags,
std::ptr::null(),
data.map_or(std::ptr::null(), |d| d.as_ptr().cast()),
)
};
if mounted != 0 {
return Err(failed_drive(drive, io::Error::last_os_error()));
return Err(io::Error::last_os_error());
}
Ok(())
}
const FSTYPE_DRIVE: &std::ffi::CStr = c"ext4";
/// The same bytes, as a path: lossless, where a round trip through `str` is
/// not.
fn as_path(path: &std::ffi::CStr) -> &std::path::Path {
use std::os::unix::ffi::OsStrExt;
std::path::Path::new(std::ffi::OsStr::from_bytes(path.to_bytes()))
}
fn ownership(path: &std::ffi::CStr) -> io::Result<(u32, u32, u32)> {
use std::os::unix::fs::MetadataExt;
let meta = std::fs::metadata(as_path(path))?;
Ok((meta.uid(), meta.gid(), meta.mode() & 0o7777))
}
fn set_ownership(path: &std::ffi::CStr, uid: u32, gid: u32, mode: u32) -> io::Result<()> {
use std::os::unix::fs::PermissionsExt;
let path = as_path(path);
std::os::unix::fs::chown(path, Some(uid), Some(gid))?;
std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode))
}
/// What the mount call is given, split out because this is the part worth
/// asserting: mounting itself needs privileges a test does not have.
@@ -400,48 +473,114 @@ fn options(share: &Mount) -> Result<(CString, CString, libc::c_ulong), Failure>
Ok((source, target, flags))
}
/// What the drive mount call is given.
/// Everything an overlay's three mounts are given, worked out before any of
/// them is attempted.
///
/// # No filesystem-specific options, and that is a decision
/// Split out because this is the part worth asserting: mounting needs
/// privileges a test does not have.
///
/// `commit=` and `barrier=` were here once and the mount failed outright:
/// *"can't mount with commit=, fs mounted w/o journal"*, `EINVAL`, and a box
/// that refused its own descriptor before the session started. Both options
/// only mean anything to a journal, and a build volume is made without one --
/// what it holds is one game, re-downloadable, mounted by a clone that is
/// destroyed with the box. Anything added here has to be an option that is
/// still true of a journal-less ext4.
/// # Where the layers go
///
/// `noatime` stays: a game reading its own install has no use for access
/// times, and writing them turns every read of a clone into a write. It is not
/// paired with `nodiratime`, which it already implies.
/// Beside the overlay, in a hidden directory named after it:
/// `/nestri/install` stacks `/nestri/.install/lower` under
/// `/nestri/.install/rw/upper`. Beside rather than under, because anything
/// mounted under `at` is covered the moment the overlay is mounted over it.
///
/// # nosuid and nodev, for the same reason every share has them
/// # Flags
///
/// What this mounts is the least trusted thing in the box: files a CDN handed
/// us, checked for the bytes the manifest named and for nothing about what
/// those bytes are. A setuid binary or a device node inside a depot is not
/// something a workload should be able to use, and no descriptor has a way to
/// ask for one.
/// `nosuid` and `nodev` on every layer and on the result, for the reason every
/// share has them: what is stacked here is files a CDN handed us, checked for
/// the bytes the manifest named and for nothing about what those bytes are.
/// **Not `noexec`** anywhere: the game's executable is in the build.
///
/// **Not `noexec`.** The game's own executable is on this volume and the whole
/// point is to run it.
fn options_drive(drive: &Drive) -> Result<(CString, CString, libc::c_ulong), Failure> {
let flags = libc::MS_NOSUID | libc::MS_NODEV | libc::MS_NOATIME;
/// `noatime` on the writable layer and the overlay, so a game reading its own
/// install does not turn every read into a write. The build image is mounted
/// read-only and has no access times to write.
///
/// No filesystem-specific options on the upper layer: `commit=` and
/// `barrier=` were once passed to a journal-less ext4 and the mount failed
/// outright with `EINVAL`.
#[derive(Debug)]
struct OverlayPlan {
lower: CString,
upper: CString,
at: CString,
lower_at: CString,
rw_at: CString,
upper_dir: CString,
work_dir: CString,
lower_flags: libc::c_ulong,
upper_flags: libc::c_ulong,
overlay_flags: libc::c_ulong,
overlay_data: CString,
}
let source = CString::new(drive.dev.as_str()).map_err(|_| {
impl OverlayPlan {
fn new(overlay: &Overlay) -> Result<Self, Failure> {
let at = std::path::Path::new(&overlay.at);
let (Some(parent), Some(name)) = (at.parent(), at.file_name()) else {
return Err(Failure::new(format!(
"the overlay mount point has no parent to put its layers beside: {:?}",
overlay.at
)));
};
let layers = parent.join(format!(".{}", name.to_string_lossy()));
let lower_at = layers.join("lower");
let rw_at = layers.join("rw");
let upper_dir = rw_at.join("upper");
let work_dir = rw_at.join("work");
// overlayfs splits its options on commas and its layer lists on
// colons, and has no escape for either that this should rely on. A
// path carrying one would mount a different directory than the one
// named, so it is refused.
for path in [&lower_at, &upper_dir, &work_dir] {
let text = path.to_string_lossy();
if text.contains([',', ':']) {
return Err(Failure::new(format!(
"the overlay mount point cannot carry a comma or a colon: {:?}",
overlay.at
)));
}
}
let data = format!(
"lowerdir={},upperdir={},workdir={}",
lower_at.display(),
upper_dir.display(),
work_dir.display()
);
let common = libc::MS_NOSUID | libc::MS_NODEV;
Ok(Self {
lower: c_string(&overlay.lower, "the build image device")?,
upper: c_string(&overlay.upper, "the writable layer device")?,
at: c_string(&overlay.at, "the overlay mount point")?,
lower_at: c_string(&lower_at.to_string_lossy(), "the overlay mount point")?,
rw_at: c_string(&rw_at.to_string_lossy(), "the overlay mount point")?,
upper_dir: c_string(&upper_dir.to_string_lossy(), "the overlay mount point")?,
work_dir: c_string(&work_dir.to_string_lossy(), "the overlay mount point")?,
lower_flags: common | libc::MS_RDONLY,
upper_flags: common | libc::MS_NOATIME,
overlay_flags: common | libc::MS_NOATIME,
overlay_data: c_string(&data, "the overlay mount point")?,
})
}
/// Which step failed, on which path, in the operating system's words.
fn failed(&self, step: &str, path: &std::ffi::CStr, error: io::Error) -> Failure {
Failure::new(format!(
"the drive device contains a nul byte: {:?}",
drive.dev
"{}: {step}: {}: {error}",
self.at.to_string_lossy(),
path.to_string_lossy()
))
})?;
let target = CString::new(drive.at.as_str()).map_err(|_| {
Failure::new(format!(
"the drive mount point contains a nul byte: {:?}",
drive.at
))
})?;
Ok((source, target, flags))
}
}
/// A nul byte inside a path is a descriptor that cannot be carried out under
/// any flags. Refused by name rather than silently emptied: an empty path turns
/// up later as a mount failure about something else entirely.
fn c_string(text: &str, what: &str) -> Result<CString, Failure> {
CString::new(text).map_err(|_| Failure::new(format!("{what} contains a nul byte: {text:?}")))
}
/// A failure names the path, which is what makes it actionable: a permission
@@ -451,10 +590,6 @@ fn failed(share: &Mount, error: io::Error) -> Failure {
Failure::new(format!("{}: {error}", share.at))
}
fn failed_drive(drive: &Drive, error: io::Error) -> Failure {
Failure::new(format!("{}: {error}", drive.at))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -578,28 +713,83 @@ mod tests {
assert_eq!(flags & libc::MS_RDONLY, 0);
}
/// The drive carries the same guard every share carries.
///
/// It is the mount that most needs it: a share is a directory this host
/// prepared, and a drive is a filesystem built out of whatever a CDN sent.
fn overlay(at: &str) -> Overlay {
Overlay {
lower: "/dev/vdb".into(),
upper: "/dev/vdc".into(),
at: at.into(),
}
}
/// The layers sit beside the overlay, never under it: anything mounted
/// under `at` is hidden the moment the overlay covers it.
#[test]
fn a_drive_is_mounted_without_devices_or_setuid_but_can_still_execute() {
let drive = Drive {
dev: "/dev/vdb".into(),
at: "/nestri/install".into(),
};
let (source, target, flags) = options_drive(&drive).unwrap();
fn an_overlays_layers_sit_beside_it_and_the_options_name_them() {
let plan = OverlayPlan::new(&overlay("/nestri/install")).unwrap();
assert_eq!(plan.lower.to_str().unwrap(), "/dev/vdb");
assert_eq!(plan.upper.to_str().unwrap(), "/dev/vdc");
assert_eq!(plan.lower_at.to_str().unwrap(), "/nestri/.install/lower");
assert_eq!(plan.rw_at.to_str().unwrap(), "/nestri/.install/rw");
assert_eq!(
source.to_str().unwrap(),
"/dev/vdb",
"the device is the source"
plan.overlay_data.to_str().unwrap(),
"lowerdir=/nestri/.install/lower,\
upperdir=/nestri/.install/rw/upper,\
workdir=/nestri/.install/rw/work"
);
}
/// Every layer carries the guard every share carries, and none of them
/// stops the game's own executable from running.
///
/// These are the mounts that most need it: a share is a directory this
/// host prepared, and a build is a filesystem made out of whatever a CDN
/// sent.
#[test]
fn every_layer_is_mounted_without_devices_or_setuid_but_can_still_execute() {
let plan = OverlayPlan::new(&overlay("/nestri/install")).unwrap();
for flags in [plan.lower_flags, plan.upper_flags, plan.overlay_flags] {
assert_eq!(flags & libc::MS_NOSUID, libc::MS_NOSUID);
assert_eq!(flags & libc::MS_NODEV, libc::MS_NODEV);
assert_eq!(flags & libc::MS_NOEXEC, 0);
}
assert_eq!(plan.lower_flags & libc::MS_RDONLY, libc::MS_RDONLY);
assert_eq!(plan.upper_flags & libc::MS_RDONLY, 0);
assert_eq!(plan.overlay_flags & libc::MS_RDONLY, 0);
assert_eq!(plan.overlay_flags & libc::MS_NOATIME, libc::MS_NOATIME);
}
/// overlayfs splits its options on commas and colons, so a path carrying
/// one would stack a different directory than the one named.
#[test]
fn an_overlay_the_kernel_would_misread_is_refused_before_anything_mounts() {
for at in [
"/nestri/in,stall",
"/nestri/in:stall",
"/",
"/nestri/ins\0tall",
] {
assert!(
OverlayPlan::new(&overlay(at)).is_err(),
"{at:?} was accepted"
);
}
}
#[test]
fn a_failed_overlay_step_names_the_overlay_the_step_and_the_path() {
let plan = OverlayPlan::new(&overlay("/nestri/install")).unwrap();
let failure = plan.failed(
"the build image",
&plan.lower_at,
io::Error::from_raw_os_error(libc::ENODEV),
);
assert!(
failure
.reason
.starts_with("/nestri/install: the build image: /nestri/.install/lower: "),
"{}",
failure.reason
);
assert_eq!(target.to_str().unwrap(), "/nestri/install");
assert_eq!(flags & libc::MS_NOSUID, libc::MS_NOSUID);
assert_eq!(flags & libc::MS_NODEV, libc::MS_NODEV);
assert_eq!(flags & libc::MS_NOATIME, libc::MS_NOATIME);
// The game's executable lives here.
assert_eq!(flags & libc::MS_NOEXEC, 0);
}
#[test]
@@ -652,7 +842,7 @@ pub mod double {
/// only thing under test.
pub struct Double {
pub mounted: Vec<Vec<Mount>>,
pub drives: Vec<Vec<Drive>>,
pub overlays: Vec<Vec<Overlay>>,
pub started: Vec<Exec>,
pub stops: usize,
pub mount_failure: Option<Failure>,
@@ -676,7 +866,7 @@ pub mod double {
fn new(exit: Exit, holds_until_stopped: bool) -> Self {
Self {
mounted: Vec::new(),
drives: Vec::new(),
overlays: Vec::new(),
started: Vec::new(),
stops: 0,
mount_failure: None,
@@ -697,8 +887,8 @@ pub mod double {
}
}
fn mount_drives(&mut self, drives: &[Drive]) -> Result<(), Failure> {
self.drives.push(drives.to_vec());
fn mount_overlays(&mut self, overlays: &[Overlay]) -> Result<(), Failure> {
self.overlays.push(overlays.to_vec());
match &self.mount_failure {
Some(failure) => Err(failure.clone()),
None => Ok(()),
apps/nesinit/tests/services_stop.rs
+3 -1
View File
@@ -50,7 +50,9 @@ fn alive(pid: i32) -> bool {
async fn a_stack_that_goes_away_takes_its_services_with_it() {
let waiters = Waiters::new();
let mut stack = Stack::from_table(waiters, SLEEPERS);
let up = stack.bring_up().expect("two sleeps did not start");
let up = stack
.bring_up(Default::default())
.expect("two sleeps did not start");
assert_eq!(up.len(), 2);
let pids = stack.pids();
build/Containerfile
+38 -28
View File
@@ -13,9 +13,9 @@
# this image is plain Arch rather than a distribution chosen for its init.
# ref(d-0064)
#
# Proton is here, and it is not a closed component: it is proton-cachyos built
# from source with --enable-wow64, which is what removes the need for a whole
# 32-bit host stack. Valve's steamclient.so is a different thing and is NOT
# Proton is here, and it is not a closed component: it is proton-ge built
# from source wow64-only, which is what removes the need for a whole 32-bit
# host stack. Valve's steamclient.so is a different thing and is NOT
# here — that one is closed, and nestri/CLAUDE.md is explicit that nothing
# closed enters this repo. Whatever layers it on top of runtime_prod is a
# closed build outside this repo — see build/README.md.
@@ -31,8 +31,7 @@
# scoped to one stage instead, `FROM ${PROTON_IMAGE}` expands to nothing, and
# the build fails with "no FROM statement found" — which says nothing about
# the actual mistake. See the Proton stage below for what this is.
ARG PROTON_IMAGE=ghcr.io/nestrilabs/proton-cachyos-native-wow64:11.0-20260703
ARG PROTON_IMAGE=ghcr.io/nestrilabs/proton-ge:11-7
# ───────────────────────────────────────────────────────────
# initial / builder
@@ -72,18 +71,14 @@ ENV ARTIFACTS=/artifacts
# ───────────────────────────────────────────────────────────
FROM builder AS mesa-build
ARG MESA_GIT=https://gitlab.freedesktop.org/mesa/mesa.git
ARG MESA_COMMIT=8ace865d958b0f17254afc427db21b0ad1747b4b
ARG MESA_GIT=https://gitlab.freedesktop.org/nowrep/mesa.git
ARG MESA_COMMIT=1af89bea491deabc4741700f8df4d41edfd22f7a
ARG JOBS=
# Our patches to the amdgpu native-context path, applied on top of the pinned
# commit. They are not cosmetic: upstream's winsys re-queries device-static
# facts on a path that costs a synchronous round trip to the host under
# virtio, and it was most of a frame's time. See each patch's own message.
#
# `git apply` and not `git am`: no committer identity is needed, and a patch
# that no longer applies stops the build here rather than producing an image
# that is quietly unpatched and slow.
COPY build/patches/mesa /build/patches/mesa
RUN test -n "$JOBS" || JOBS=$(nproc) && \
@@ -173,14 +168,13 @@ RUN --mount=type=cache,target=/root/.cargo/registry \
# `FROM scratch` over the tree, so its root *is* the tree and there is nothing
# in it to run — only something to copy from.
#
# Built with `--enable-wow64`, which is the whole reason it is a build of ours
# rather than the distribution's package. wow64 runs 32-bit Windows code
# inside a 64-bit unix process, so a box needs no lib32 anything: no 32-bit
# glibc, no second Mesa for i686, and — the one that matters most here — no
# Built wow64-only, which is the whole reason it is a build of ours rather
# than a released one. wow64 runs 32-bit Windows code inside a 64-bit unix
# process, so a box needs no lib32 anything: no 32-bit glibc, no second Mesa
# for i686, and — the one that matters most here — no
# second capture layer, because the game is a 64-bit process and loads the
# 64-bit Vulkan loader the existing layer already sits in. The distribution's
# package is built without the flag, which is exactly why it depends on
# lib32-*.
# 64-bit Vulkan loader the existing layer already sits in. The released builds
# carry a 32-bit unix side, which is exactly why they need lib32-*.
#
# Override to build it yourself; the tag is a version and moves deliberately.
# ───────────────────────────────────────────────────────────
@@ -269,7 +263,7 @@ RUN pacman -Syu --noconfirm --needed \
libglvnd libdrm libepoxy libxxf86vm libinput wayland \
expat zlib llvm-libs lm_sensors elfutils libva shaderc vulkan-icd-loader \
pixman libxkbcommon xcb-util-keysyms xorg-xwayland \
pipewire pipewire-audio wireplumber opus \
pipewire pipewire-audio pipewire-pulse libpulse wireplumber opus \
python libunwind \
&& rm -f /usr/share/libalpm/hooks/dbus-reload.hook \
&& pacman -Rdd --noconfirm systemd systemd-sysvcompat \
@@ -281,6 +275,15 @@ RUN pacman -Syu --noconfirm --needed \
# after the prefix had already been created -- so the session got past every
# check that Proton was present and usable.
#
# `pipewire-pulse` and `libpulse` are Wine's audio. Its driver speaks only the
# PulseAudio protocol, so without the translation layer a game starts, renders
# and plays silently, with nothing failing to say why. `libpulse` is what that
# driver, `winepulse.so`, links, and it would arrive as a dependency of
# pipewire-pulse anyway. It is named because the dependency is Wine's and not
# pipewire-pulse's, and a library that only arrives as somebody else's
# dependency goes away when they do. The check below covers `winepulse.so`, so a
# missing library fails the build rather than the session.
# `python` is not a build dependency here -- the builder stage has its own for
# Mesa -- it is a *runtime* one. The compatibility tool's entry point is a
# Python script, so a box without an interpreter starts a game and the launch
@@ -368,7 +371,7 @@ COPY --from=mesa-build /artifacts/mesa /
COPY --from=nestri-build /artifacts/nestri /
# The Proton tree, whose image root is the tree, so this lands it at
# /usr/share/steam/compatibilitytools.d/proton-cachyos.
# /usr/share/steam/compatibilitytools.d/proton-ge.
#
# Deliberately not in the strip manifest above: that list is our own build
# output, and the two stripping decisions are not the same one. Proton ships
@@ -470,11 +473,17 @@ RUN for intruder in /usr/lib/systemd/systemd /sbin/openrc-init /usr/bin/openrc-i
# directory. Forty files, every one a false positive, hiding the one real
# entry.
#
# So: the programs in `bin/`, which are ordinary ELF and resolve normally, and
# So: the programs in `bin/`, which are ordinary ELF and resolve normally,
# `ntdll.so`, which is the first thing Wine loads and the one that linked the
# missing library. That is exactly the failure that got through, with none of
# the noise. The Windows-side DLLs beside them are not ELF and `ldd` skips them
# anyway.
# missing library, and `winepulse.so`, which is audio. That is exactly the
# failure that got through, with none of the noise. The Windows-side DLLs beside
# them are not ELF and `ldd` skips them anyway.
#
# Each file is resolved with its own directory on the library path. That is
# where Wine's loader finds the siblings a unix object links by soname, so
# `winepulse.so` naming `ntdll.so` resolves the way it does at run time, and
# what is left is a library the image really lacks. For every other file in
# the list it changes nothing.
# The output is one file per line with its own missing libraries under it, and
# then every missing library once at the end. That last list is what somebody
# acts on -- it is the set of packages to add -- and forty files each naming the
@@ -486,13 +495,14 @@ RUN failed=0; \
: > /tmp/missing-libs; \
for f in /usr/bin/nesinit /usr/bin/nescope /usr/bin/neshub /usr/bin/neswire \
/usr/lib/libnescapture_layer.so \
/usr/bin/dbus-daemon /usr/bin/pipewire /usr/bin/wireplumber /usr/bin/ip \
/usr/bin/dbus-daemon /usr/bin/pipewire /usr/bin/pipewire-pulse \
/usr/bin/wireplumber /usr/bin/ip \
/usr/lib/libgallium-*.so /usr/lib/libEGL_mesa.so.0 \
/usr/lib/libvulkan_*.so /usr/lib/dri/*.so /usr/lib/gbm/*.so \
/usr/share/steam/compatibilitytools.d/proton-cachyos/files/bin/* \
/usr/share/steam/compatibilitytools.d/proton-cachyos/files/lib*/wine/*-unix/ntdll.so; do \
[ -e "$f" ] || continue; \
libs="$(ldd "$f" 2>/dev/null | awk '/not found/ { print $1 }')"; \
libs="$(LD_LIBRARY_PATH="${f%/*}" ldd "$f" 2>/dev/null | awk '/not found/ { print $1 }')"; \
[ -n "$libs" ] || continue; \
failed=1; \
printf ' %s\n' "$f" >&2; \
@@ -507,8 +517,8 @@ RUN failed=0; \
fi
RUN for required in /usr/bin/nesinit /usr/bin/nescope /usr/bin/neshub /usr/bin/neswire \
/usr/bin/dbus-daemon /usr/bin/pipewire /usr/bin/wireplumber /usr/bin/ip \
/usr/bin/python3 \
/usr/bin/dbus-daemon /usr/bin/pipewire /usr/bin/pipewire-pulse \
/usr/bin/wireplumber /usr/bin/ip /usr/bin/python3 \
/usr/share/steam/compatibilitytools.d/proton-cachyos/proton; do \
test -x "$required" || { echo "the image is missing $required" >&2; exit 1; }; \
done
build/Containerfile.proton
+16 -93
View File
@@ -1,102 +1,25 @@
# ═══════════════════════════════════════════════════════════
# proton-cachyos, built wow64-only
# proton-ge, built wow64-only, as a copyable image
#
# This file does not build Proton. It packages a Proton tree that was already
# built, and it cannot build it: proton-ge's build runs its own steps in the
# Steam Runtime SDK image through a container engine, so it runs on the host,
# and running it inside this build would mean nested containers.
# `scripts/proton-build.sh` does the build, and this file wraps the result.
#
# Separate from the guest Containerfile because it is a separate cadence:
# hours to build, and only when PROTON_TAG moves. The guest image pulls the
# result from a registry instead of rebuilding it, which is why this file is
# not part of that build and is not reached by `make build`.
# result from a registry instead of rebuilding it, which is why none of this
# is reached by `make build`.
#
# The final stage is FROM scratch, so the image *is* the Proton tree — nothing
# to run, only something to COPY --from.
# It is FROM scratch, so the image *is* the Proton tree. There is nothing in
# it to run, only something to COPY --from.
#
# Build and publish with `make proton-image` / `make proton-push`. **Its
# context is this directory**, not the repository root the guest build uses:
# all it needs is the two scripts beside it, and a context of the whole tree
# would hand it a multi-gigabyte `output/` for no reason.
#
# `PROTON_TAG` is the one thing to change, and the Makefile derives the
# published image's version from it. They are the same number in two
# spellings, and an image whose name does not say which Proton is inside it
# is worse than no image.
# ═══════════════════════════════════════════════════════════
FROM docker.io/archlinux:base-devel AS builder
# proton-cachyos-native's makedepends, minus every lib32-* (that is the whole
# point of --enable-wow64), with two substitutions Arch requires: ocl-icd
# provides opencl-icd-loader, and mesa-libgl is folded into libglvnd.
#
# glib2-devel is the one that is easy to miss — it carries glib-mkenums, which
# libsoup's meson looks up through glib-2.0's pkg-config variables and fails on
# obscurely. unzip and zip are for the xalia dist step. afdko is not packaged at
# all, and the fonts submodule needs it, so it comes from PyPI.
RUN pacman -Syu --noconfirm --needed \
alsa-lib clang cmake curl ffmpeg fontforge giflib git glib2-devel \
glslang gnutls gtk3 libgphoto2 libglvnd libpulse libva libxcomposite \
libxinerama libxxf86vm lld mesa meson ninja nasm \
opencl-headers ocl-icd pcsclite perl perl-json python python-pip \
python-pefile python-setuptools-scm rsync rust samba unixodbc \
unzip zip v4l-utils vulkan-headers vulkan-icd-loader wayland \
wayland-protocols wget xorg-util-macros \
mingw-w64-gcc mingw-w64-binutils mingw-w64-crt mingw-w64-headers \
mingw-w64-winpthreads \
&& pip install --break-system-packages --no-cache-dir afdko \
&& pacman -Scc --noconfirm
# widl generates vkd3d's public headers. Without it autoconf sets HAVE_WIDL to
# false, the headers are silently not generated, and the build dies an hour
# later on a missing vkd3d_d3dx9shader.h. Arch ships widl only inside `wine`,
# which requires multilib; the AUR's mingw-w64-tools builds it standalone, so do
# the same.
#
# The digest is not a formality. What is extracted here has its `configure` and
# its makefiles run as root in this builder, and what they produce is copied
# into the image a box runs -- so whoever can change these bytes can change what
# runs on every host. SourceForge hands the request to whichever mirror it
# likes, over a connection this builder does not pin, and the project publishes
# no signature. The digest is the only thing that makes the mirror not matter.
#
# Taken 2026-09-14 from two different mirrors of v14.0.0, which agreed. A
# mismatch here is not a thing to paper over by taking the new digest: it means
# the bytes behind this exact version string changed, and that wants looking at
# before it wants fixing.
ARG MINGW_W64_VER=14.0.0
ARG MINGW_W64_SHA256=6eaf921d9eb987d3820b364ea9775bc19b965ec81490b6fdd716526c28e1995c
RUN curl -fL --retry 3 -o /tmp/mingw-w64.tar.bz2 \
"https://sourceforge.net/projects/mingw-w64/files/mingw-w64/mingw-w64-release/mingw-w64-v${MINGW_W64_VER}.tar.bz2/download" \
&& echo "${MINGW_W64_SHA256} /tmp/mingw-w64.tar.bz2" | sha256sum -c - \
&& tar xf /tmp/mingw-w64.tar.bz2 -C /tmp \
&& for arch in i686-w64-mingw32 x86_64-w64-mingw32; do \
mkdir -p "/tmp/widl-${arch}" && cd "/tmp/widl-${arch}" \
&& "/tmp/mingw-w64-v${MINGW_W64_VER}/mingw-w64-tools/widl/configure" \
--prefix=/usr --target="${arch}" --program-prefix="${arch}-" \
&& make -j"$(nproc)" && make install; \
done \
&& rm -rf /tmp/mingw-w64* /tmp/widl-*
WORKDIR /build
ARG PROTON_GIT=https://github.com/CachyOS/proton-cachyos.git
ARG PROTON_TAG=cachyos-11.0-20260703-native
ARG GECKO_VER=2.47.4
ARG MONO_VER=11.2.0
ARG XALIA_VER=0.4.9
ARG JOBS=
# Fetch and build are separate layers on purpose: the submodule checkout runs
# well past ten minutes, and a build that fails on a flag or a missing tool must
# not pay for it again.
COPY scripts/proton-fetch.sh /build/proton-fetch.sh
RUN PROTON_GIT="${PROTON_GIT}" PROTON_TAG="${PROTON_TAG}" \
GECKO_VER="${GECKO_VER}" MONO_VER="${MONO_VER}" XALIA_VER="${XALIA_VER}" \
bash /build/proton-fetch.sh
COPY scripts/proton-build.sh /build/proton-build.sh
RUN GECKO_VER="${GECKO_VER}" MONO_VER="${MONO_VER}" JOBS="${JOBS}" \
bash /build/proton-build.sh
# ═══════════════════════════════════════════════════════════
# The publishable artifact: the Proton tree and nothing else
# context is the built tree** (`output/proton/obj/dist`), and not this
# directory or the repository root: the tree is the whole input, and it is
# the only thing the build needs to send.
# ═══════════════════════════════════════════════════════════
FROM scratch
COPY --from=builder /artifacts/proton/ /
ARG BUILD_NAME=proton-ge
COPY . /usr/share/steam/compatibilitytools.d/${BUILD_NAME}/
build/Containerfile.proton.containerignore
-15
View File
@@ -1,15 +0,0 @@
# This build's context is `build/`, not the repository root, because all it
# needs is the two scripts beside the Containerfile.
#
# `output/` is the reason this file exists. It holds packed rootfs images —
# multiple gigabytes each — and a build context is copied before the first
# instruction runs, so without this line every Proton build starts by moving
# the last one it produced.
output
# Nothing else here is an input to this build.
etc
README.md
Makefile
Containerfile
Containerfile.containerignore
build/Makefile
+83 -12
View File
@@ -1,5 +1,5 @@
SHELL := /bin/bash
.PHONY: build build-debug image image-debug proton-image proton-push clean help
.PHONY: build build-debug image image-debug kernel kernel-clean proton-image proton-push proton-clean clean help
CONTAINER_RT := $(shell command -v podman 2>/dev/null || command -v docker 2>/dev/null)
ifeq ($(CONTAINER_RT),)
@@ -37,14 +37,53 @@ IMAGE_NAME := ghcr.io/nestrilabs/nestri/base
# number in two spellings and an image whose name does not say which Proton is
# inside it is worse than no image at all. Changing the tag by hand and
# forgetting the version is exactly the mistake this removes.
PROTON_TAG ?= cachyos-11.0-20260703-native
PROTON_VERSION := $(PROTON_TAG:cachyos-%-native=%)
PROTON_GIT ?= https://github.com/gloriouseggroll/proton-ge-custom.git
PROTON_TAG ?= GE-Proton11-7
PROTON_VERSION := 11.0-20260703
PROTON_IMAGE ?= ghcr.io/nestrilabs/proton-cachyos-native-wow64
PROTON_REF := $(PROTON_IMAGE):$(PROTON_VERSION)
# The directory name under compatibilitytools.d, and the name Steam shows.
PROTON_NAME := proton-ge
OUTPUT_DIR := output
# Source, build tree, ccache and cargo downloads. Tens of gigabytes, and it
# lives under output/ because that directory is already ignored by git and
# kept out of the guest build's context.
PROTON_WORK := $(OUTPUT_DIR)/proton
ROOTFS_SIZE ?= 3G
FORCE_REBUILD ?=
# The guest kernel: CachyOS's fork, for its scheduler patches, with our own
# minimal config on top (kernel/nestri.fragment). The tree is a clone rather
# than vendored, and lives under output/ for the same reason Proton's does.
# KERNEL_SRC may point at an existing tree elsewhere.
KERNEL_GIT ?= https://github.com/CachyOS/linux.git
KERNEL_REF ?= cachyos-7.2.6-1
# The target hardware is at least AVX2-capable. Never `native`: the build host
# is not the target. Empty leaves the tree's own generic choice alone.
KERNEL_MARCH ?= x86-64-v3
# Experimental: KERNEL_INFINITY=1 applies the Infinity scheduler series (fair,
# rt and DRM scheduler rework) on top. It builds in its own tree and installs
# under its own name, so the stock kernel is never patched and switching the
# flag back and forth never needs a revert.
#
# The series is published per CachyOS release, so its directory is derived
# from KERNEL_REF rather than written twice: bumping the kernel to a release
# the series does not cover fails at the patch step, not in a booted box.
KERNEL_INFINITY ?=
INFINITY_GIT ?= https://github.com/galpt/infinity-sched-new.git
INFINITY_REV ?= e6c85d841f25e21393a7ea47fdef286318915d8c
INFINITY_SERIES := patches/cachyos/tuned-eevdf/$(KERNEL_REF:cachyos-%=%)
INFINITY_WORK := $(OUTPUT_DIR)/infinity-sched
ifneq ($(KERNEL_INFINITY),)
KERNEL_SRC ?= $(OUTPUT_DIR)/kernel-infinity
KERNEL_OUTPUT := $(OUTPUT_DIR)/vmlinux-infinity
else
KERNEL_SRC ?= $(OUTPUT_DIR)/kernel
KERNEL_OUTPUT := $(OUTPUT_DIR)/vmlinux
endif
build:
DOCKER_BUILDKIT=1 $(CONTAINER_RT) build $(if $(FORCE_REBUILD),--no-cache,) \
--build-arg PROTON_IMAGE=$(PROTON_REF) \
@@ -55,13 +94,20 @@ build-debug:
--build-arg PROTON_IMAGE=$(PROTON_REF) \
-f Containerfile -t $(IMAGE_NAME):debug --target runtime_debug $(CONTEXT)
# Hours, and only when PROTON_TAG moves. Its context is this directory rather
# than the repository root: the two scripts beside the Containerfile are the
# whole input, and the root would hand it everything else for nothing.
# Hours, and only when PROTON_TAG moves. Two steps: the build runs on the host,
# because proton-ge's build drives the container engine itself, and the
# finished tree is then the whole context of a FROM scratch image. See the head
# of Containerfile.proton for why the build cannot run inside it.
#
# A second run of the same tag resumes where the last one stopped. A new tag,
# or FORCE_REBUILD, starts the tree over but keeps ccache.
proton-image:
DOCKER_BUILDKIT=1 $(CONTAINER_RT) build $(if $(FORCE_REBUILD),--no-cache,) \
--build-arg PROTON_TAG=$(PROTON_TAG) \
-f Containerfile.proton -t $(PROTON_REF) .
PROTON_GIT=$(PROTON_GIT) PROTON_TAG=$(PROTON_TAG) PROTON_WORK=$(PROTON_WORK) \
BUILD_NAME=$(PROTON_NAME) CONTAINER_ENGINE=$(notdir $(CONTAINER_RT)) \
FORCE_REBUILD=$(FORCE_REBUILD) \
bash scripts/proton-build.sh
$(CONTAINER_RT) build --build-arg BUILD_NAME=$(PROTON_NAME) \
-f Containerfile.proton -t $(PROTON_REF) $(PROTON_WORK)/obj/dist
@echo "Built $(PROTON_REF)"
# Publishing is what makes `make build` cheap for everyone else, since that
@@ -82,8 +128,29 @@ image-debug: build-debug
@mkdir -p $(OUTPUT_DIR)
bash scripts/mkimage.sh $(IMAGE_NAME):debug $(OUTPUT_DIR)/rootfs-debug.ext4 $(ROOTFS_SIZE)
# Not part of `build`: it changes far less often than the rootfs, and nothing
# in the image depends on it. A rerun rebuilds only what changed in the tree.
kernel:
KERNEL_GIT=$(KERNEL_GIT) KERNEL_REF=$(KERNEL_REF) KERNEL_SRC=$(KERNEL_SRC) \
KERNEL_MARCH=$(KERNEL_MARCH) KERNEL_OUTPUT=$(KERNEL_OUTPUT) \
KERNEL_INFINITY=$(KERNEL_INFINITY) INFINITY_GIT=$(INFINITY_GIT) \
INFINITY_REV=$(INFINITY_REV) INFINITY_SERIES=$(INFINITY_SERIES) \
INFINITY_WORK=$(INFINITY_WORK) \
bash scripts/kernel-build.sh
# Leaves $(PROTON_WORK) and the kernel alone: one is hours of build and ccache,
# the other a clone and a warm object tree, and a rootfs is neither.
# proton-clean and kernel-clean are the ones that drop them.
clean:
rm -rf $(OUTPUT_DIR)
find $(OUTPUT_DIR) -mindepth 1 -maxdepth 1 ! -name proton ! -name 'kernel*' \
! -name 'vmlinux*' ! -name infinity-sched -exec rm -rf {} + 2>/dev/null || true
kernel-clean:
rm -rf $(OUTPUT_DIR)/kernel $(OUTPUT_DIR)/kernel-infinity $(INFINITY_WORK) \
$(OUTPUT_DIR)/vmlinux $(OUTPUT_DIR)/vmlinux-infinity
proton-clean:
rm -rf $(PROTON_WORK)
help:
@echo "Usage:"
@@ -91,9 +158,13 @@ help:
@echo " make build-debug Build the runtime_debug container image"
@echo " make image Build + pack runtime_prod into output/rootfs.ext4"
@echo " make image-debug Build + pack runtime_debug into output/rootfs-debug.ext4"
@echo " make clean Remove output/"
@echo " make kernel Build the guest kernel into output/vmlinux"
@echo " make KERNEL_INFINITY=1 kernel Experimental Infinity scheduler build → output/vmlinux-infinity"
@echo " make kernel-clean Remove both kernel trees and images"
@echo " make clean Remove the rootfs images from output/"
@echo " make proton-image Build Proton from source (hours)"
@echo " make proton-clean Remove the Proton source, build tree and ccache"
@echo " make proton-push Build it and publish it"
@echo " make FORCE_REBUILD=1 ... Rebuild from scratch, no layer cache"
@echo " make PROTON_TAG=... ... Use a different proton-cachyos tag"
@echo " make PROTON_TAG=... ... Use a different proton-ge tag"
@echo " make ROOTFS_SIZE=8G image Override the packed image size (default 5G)"
build/README.md
+94 -26
View File
@@ -66,17 +66,16 @@ wrong about Proton: it is compiled from source, which is not a thing you can
do with closed software. Keeping it out cost a box the only way it has to run
a Windows title, for a rule that did not apply to it.
What it is: **proton-cachyos built with `--enable-wow64`**, pulled by tag as a
What it is: **proton-ge built wow64-only**, pulled by tag as a
published image rather than rebuilt here, because it takes hours and moves
only when its own tag does. `PROTON_IMAGE` overrides the tag, and it has to be
declared before the first `FROM` — an `ARG` a `FROM` expands is global or it
declared before the first `FROM`: an `ARG` a `FROM` expands is global or it
is nothing, and getting that wrong fails with `no FROM statement found`, which
says nothing about the actual mistake. wow64 is the whole reason it is a build of ours
and not the distribution's package — it runs 32-bit Windows code inside a
64-bit unix process, so a box needs no lib32 glibc, no second Mesa for i686,
and no second capture layer for 32-bit titles to be captured. The
distribution's package is built without the flag, which is exactly why it
depends on `lib32-*`.
says nothing about the actual mistake. wow64 is the whole reason it is a build
of ours and not a released one. It runs 32-bit Windows code inside a 64-bit
unix process, so a box needs no lib32 glibc, no second Mesa for i686, and no
second capture layer for 32-bit titles to be captured. The released builds
carry a 32-bit unix side, which is exactly why they need `lib32-*`.
It costs about 1.4 GB of image, and it is the one thing in here that is
payload-shaped: a compatibility layer for Windows games in an image that is
@@ -92,6 +91,59 @@ the virtio-gpu native-context protocol never drift apart. Only Mesa —
`virglrenderer` is the host half of that protocol and nesbox builds its own,
patched, from `nesbox/patches/`; nothing in this image carries it.
## The guest kernel
```sh
make kernel # clone if needed, configure, verify, build → output/vmlinux
make KERNEL_SRC=~/src/linux kernel # build an existing tree instead
make kernel-clean # drop the tree and the image
```
CachyOS's fork (`KERNEL_REF` in the `Makefile`), taken for its scheduler
patches, not its config: theirs is a desktop build with thousands of modules,
and this guest has `CONFIG_MODULES` off and no `/lib/modules` at all.
- **`kernel/nestri.fragment` is the source of truth**, and says why each entry
is there. It is merged onto the tree's `.config`, resolved with
`olddefconfig`, and then **checked**: any entry that did not survive fails the
build. `merge_config.sh` and `olddefconfig` both drop options quietly, and
the worst of these fails as perfectly healthy, perfectly silent audio.
- **`kernel/base.config` is only a seed** for a tree with no `.config`, so a
fresh clone does not start from `defconfig`'s enormous driver set. Change the
fragment, not the seed and not a tree's `.config`.
- **`vmlinux`, not `bzImage`.** The guest is loaded as a raw ELF with no
bootloader in the path. It is ~16 MB unstripped, which costs nothing at run
time: only the loadable segments are mapped.
- **`-march=x86-64-v3`** goes in through `KCFLAGS`. It is safe in a kernel:
the kernel's own `-mno-sse -mno-avx …` masks every vector extension off
whatever the flag order, leaving v3's integer ISA.
- The tree is off the pinned ref (a bisect, a local patch)? The build warns
and builds what is there rather than checking the ref out over your work.
### Experimental: the Infinity scheduler
```sh
make KERNEL_INFINITY=1 kernel # → output/vmlinux-infinity
```
Applies [infinity-sched](https://github.com/galpt/infinity-sched-new)'s
series (GPL-2), which reworks the fair, RT and DRM schedulers for latency
under load. It is pinned by commit (`INFINITY_REV`), and the series directory
comes from `KERNEL_REF`, since upstream publishes one per CachyOS release.
- **Its own tree and its own image.** It builds in `output/kernel-infinity`,
so the stock kernel is never patched and switching between the two needs no
revert.
- **All or nothing, zero fuzz.** The whole series is checked against the
stacked result before any of it is applied. The applied commit is recorded in
the tree; to move `INFINITY_REV`, start over with `make kernel-clean`.
- **Only the CPU half does anything here.** virtio-gpu does not use the DRM
scheduler and `CONFIG_DRM_SCHED` is not built, so the GPU patch is compiled
out. It is applied anyway because upstream says a partial series
misbehaves.
- Upstream's `/sys/kernel/debug/infinity_*` counters need `CONFIG_DEBUG_FS`,
which this kernel does not have.
## Two packages that look droppable and are not
`llvm-libs` is 164 MB, the largest single thing in the image after Proton, and
@@ -109,13 +161,14 @@ a package list says that; the check below is what said it.
`make build` **pulls** Proton by tag; it does not build it. Building it takes
hours and it changes only when its tag moves, so it is one image published
once and copied into every guest image after that. `Containerfile.proton` is
once and copied into every guest image after that. `make proton-image` is
that build, and it lives here so the published tag stays reproducible from
this tree rather than from somebody's laptop.
```sh
make proton-image # the current tag
make PROTON_TAG=cachyos-11.1-20261115-native proton-image
make proton-image # the current tag
make PROTON_TAG=GE-Proton11-8 proton-image
make proton-clean # drop the source, build tree and ccache
```
**`PROTON_TAG` is the only thing to change.** The published version is derived
@@ -125,23 +178,38 @@ which Proton is inside it is worse than no image. The `Containerfile`'s own
`PROTON_IMAGE` default is a fallback for a bare container build; going through
`make` is what keeps them in step.
Its **context is `build/`**, not the repository root the guest build uses. All
it needs is the two scripts beside it, and `Containerfile.proton.containerignore`
keeps `output/` out of that context — a build context is copied before the
first instruction runs, so without it every Proton build would begin by moving
the last rootfs image it produced.
**The build runs on the host, not in a `podman build`.** proton-ge's build is
container-driven itself: `make` runs outside, and every step runs in the Steam
Runtime SDK image through the container engine, so the host needs only git,
make and podman. Wrapping that in a container build would mean nested
containers. So there are two steps:
Two things in the recipe are worth knowing before changing it:
1. `scripts/proton-build.sh` clones the tag with its submodules, applies
proton-ge's patch set, and runs its build with one change: the arch list
drops the 32-bit unix side, which is what makes it wow64-only. Everything
happens under `output/proton/`, which is tens of gigabytes.
2. `Containerfile.proton` is `FROM scratch` with the built tree as its whole
context, so the image is the tree and nothing else.
- **Fetch and build are separate layers on purpose.** The submodule checkout
runs well past ten minutes, and a build that fails on a flag or a missing
tool must not pay for that again. Keep anything that can fail *fast* in
`proton-build.sh`.
- **`widl` is built by hand from the mingw-w64 release.** Without it autoconf
quietly sets `HAVE_WIDL` to false, vkd3d's public headers are never
generated, and the build dies an hour later on a missing header. Arch ships
`widl` only inside `wine`, which wants multilib — which is the thing
`--enable-wow64` exists to avoid.
Things worth knowing before changing it:
- **A rerun of the same tag resumes.** The clone, the patching and the
configure step each run once per tag, and proton-ge's own make picks up
where it stopped. A new tag or `FORCE_REBUILD=1` starts the tree over;
ccache and the cargo downloads survive both. `make clean` leaves all of it
alone, and `make proton-clean` removes it.
- **The patch script does not fail on a patch that does not apply.** It
carries on and exits 0, so `proton-build.sh` greps its output
(`output/proton/patch.log`) and stops. Otherwise the result is an image
that looks fine and is missing a fix.
- **`patches/proton-ge/` is ours, applied after proton-ge's own set.** It holds
what wow64-only needs that proton-ge's makefile does not handle, and fixes
for things a tag pinned that have since moved. Each patch says why it exists
at its top, and each one fails the build outright once it stops applying,
which on a tag bump is usually upstream having fixed it.
- **The patch script is not idempotent**, so a tree is patched exactly once.
An interrupted run resets every submodule to the commits the tag pins before
patching again.
## There is no init system in here, and that is the design
build/etc/pipewire/pipewire-pulse.conf.d/50-nestri.conf
+66
View File
@@ -0,0 +1,66 @@
# pipewire-pulse, the PulseAudio protocol on top of PipeWire, for clients that
# speak nothing else. Wine's audio driver is one of them.
#
# The socket moves out of the service user's runtime directory. That directory
# is 0700 and the workload runs as a different user, so a client there finds
# nothing and plays silently, which is the failure PipeWire's own socket was
# moved for. The workload is pointed here with PULSE_SERVER, and nesinit starts
# this under a umask that lets a user who does not own the socket open it. This
# path is written in nesinit too, and a test there compares the two.
#
# A drop-in replaces a key of pulse.properties rather than appending to it, so
# the default "unix:native" socket is not created as well.
pulse.properties = {
server.address = [ "unix:/run/pipewire/pulse-native" ]
# module-always-sink creates a "Dummy Output" sink whenever no other sink
# exists, and while neswire's sink is not up yet that is always. A second
# sink is a second default-node candidate, and a game that picks it plays
# into nothing. neswire has to be the only sink; see the WirePlumber
# drop-in.
pulse.cmd.always-sink = false
# The root is read-only and every box starts from the same image, so there
# is nothing to restore and nowhere to save, which is the reasoning behind
# turning off WirePlumber's state hooks. Left on, these are one more place
# that could pin a default.
pulse.cmd.device-manager = false
pulse.cmd.device-restore = false
pulse.cmd.stream-restore = false
# Wine sizes its whole audio path from this number. At startup its driver
# opens a probe stream asking for a 1-frame request size, reads back what
# the server grants, which is this floor, and makes the device period ten
# times that. Streams then queue three periods. The default floor of 256
# frames therefore becomes a 53 ms period and about 160 ms of audio in
# flight, which is desktop playback and not a game being streamed.
#
# 48 frames (1 ms) makes that period 10 ms, the default period Windows
# itself uses, so a game gets the buffering it was written for. A stream
# then asks for 480-frame requests, which is what is granted, so the floor
# affects the probe and nothing else Wine does.
pulse.min.req = 48/48000
# What a client gets when it asks for no particular buffering. The shipped
# default is two seconds, which is only harmless where nobody is listening
# live. 10 ms requests and 40 ms of target buffer are enough for a client
# that does not say.
pulse.default.req = 480/48000
pulse.default.tlength = 1920/48000
}
# pipewire-pulse.conf raises the minimum node latency to 1024 frames (21.3 ms)
# inside a virtual machine, which a box is. That puts every Pulse client, and
# therefore Wine, at least 21 ms behind, whatever it asked for. Floor it where
# the graph itself is floored instead; see the pipewire.conf drop-in for why
# 256 frames. A drop-in appends to this array, and every matching rule applies
# in order, so this runs after the shipped rule and wins.
pulse.properties.rules = [
{ matches = [ { cpu.vm.name = !null } ]
actions = {
update-props = {
pulse.min.quantum = 256/48000
}
}
}
]
build/etc/pipewire/pipewire.conf.d/50-nestri.conf
+24
View File
@@ -0,0 +1,24 @@
# The graph's clock, floored where the audio sender needs it.
#
# Inside a virtual machine, pipewire.conf raises the graph's minimum quantum to
# 1024 frames (21.3 ms), because a VM's timers jitter and small quanta then
# underrun on real audio hardware. A box is a VM, so that rule applies, and it
# silently overrides neswire's request for its 5 ms packet size. Every sample
# then waits for a 21 ms cycle before it can be encoded.
#
# The floor comes down to 256 frames (5.3 ms), the smallest power of two that
# holds one of neswire's packets. It is not lower than that on purpose: nothing
# here needs less, and the reason for the rule, a VM's timer driving the graph,
# still holds, because the Dummy-Driver is exactly that.
#
# A drop-in appends to a rules array, and every matching rule applies in order,
# so this one runs after the shipped one and wins.
context.properties.rules = [
{ matches = [ { cpu.vm.name = !null } ]
actions = {
update-props = {
default.clock.min-quantum = 256
}
}
}
]
build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf
+7 -8
View File
@@ -3,14 +3,13 @@
# The goal is that `neswire` is the *only* Audio/Sink in the graph, so
# default-node selection has exactly one candidate and cannot pick wrong.
#
# The "Dummy Output" (`auto_null`) sink is not disabled here, and no longer
# needs to be: it came from pipewire-pulse's `module-always-sink`, and
# pipewire-pulse is no longer installed. WirePlumber ships
# scripts/fallback-sink.lua, which creates a node by the same name, but no
# component in wireplumber.conf references it, so it never loads. If auto_null
# ever comes back, it came back with pipewire-pulse -- the switch is
# `pulse.cmd.always-sink = false` in a pipewire-pulse.conf.d drop-in, not
# anything on this side.
# The "Dummy Output" (`auto_null`) sink is not disabled here, because it does
# not come from this side. pipewire-pulse creates it with `module-always-sink`,
# and the pipewire-pulse drop-in turns that off with
# `pulse.cmd.always-sink = false`. WirePlumber ships scripts/fallback-sink.lua,
# which creates a node by the same name, but no component in wireplumber.conf
# references it, so it never loads. If auto_null appears, look at the
# pipewire-pulse drop-in first.
#
# pipewire.conf's Dummy-Driver / Freewheel-Driver are a third thing again:
# support.node.driver objects with no ports -- drivers, not sinks, so nothing
build/kernel/base.config
+3147
View File
File diff suppressed because it is too large Load Diff
build/kernel/nestri.fragment
+207
View File
@@ -0,0 +1,207 @@
# Kconfig fragment for the guest kernel.
#
# Merged onto the tree's config by scripts/kernel-build.sh with the kernel's own
# merge_config.sh, then resolved with `make olddefconfig`, then checked: every
# entry below must hold in the final .config or the build stops. A fragment
# rather than a saved full config: a full .config pins thousands of symbols and
# rots the moment the tree is bumped, while this says only what the guest
# actually requires and lets the version's own defaults handle the rest.
#
# Every entry here is load-bearing or a deliberate performance choice. Nothing
# is speculative.
# ── Gaming needs ─────────────────────────────────────────
# NTSYNC is a great improvement over fsync and esync approaches.
CONFIG_NTSYNC=y
# ── Timers ───────────────────────────────────────────────
# The one that cost a day of silent audio. The guest has no sound hardware, so
# PipeWire drives its whole graph off a timerfd at a 2.67ms cycle, which a
# HZ_PERIODIC/250 kernel cannot express. Symptom: everything reports healthy,
# neswire sends a steady ~3 kbps of Opus, and every sample is zero.
CONFIG_HIGH_RES_TIMERS=y
CONFIG_NO_HZ_IDLE=y
CONFIG_HZ_1000=y
# ── Shared memory ────────────────────────────────────────
# PipeWire moves audio through memfd, not through its socket. Wine uses POSIX
# shm and semaphores directly. /dev/shm itself is mounted by nesinit --
# devtmpfs does not provide it.
CONFIG_SHMEM=y
CONFIG_MEMFD_CREATE=y
CONFIG_TMPFS=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# ── What PipeWire schedules on ───────────────────────────
# FUTEX_PI is also what Proton's fsync uses.
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
# ── Clock sources ────────────────────────────────────────
# Listed because they were correct throughout the silent-audio episode. A good
# clock *source* says nothing about whether the kernel can schedule a timer
# *event* at the resolution asked for; only the block above governs that.
CONFIG_KVM_GUEST=y
CONFIG_PARAVIRT_CLOCK=y
CONFIG_X86_TSC=y
# ── Steal time ───────────────────────────────────────────
# Time the host spends running something else on a vCPU is subtracted from
# whatever task was on it, rather than charged to it. vCPUs share a set of host
# cores instead of owning one each, so steal happens, and without this the
# scheduler reads it as the task having run -- a frame thread preempted by the
# host looks, to the fair class, like one that used its slice.
CONFIG_PARAVIRT_TIME_ACCOUNTING=y
# ── Idle: poll before halting ────────────────────────────
# An idle vCPU that halts exits to the host, and waking it costs an IPI and a
# VM entry. The frame pipeline sleeps and wakes every few milliseconds, so that
# is paid constantly. haltpoll spins for a bounded, adaptive window first
# (guest_halt_poll_ns, 200us), so a wakeup that comes quickly never leaves the
# guest. When the host offers it, the guest also switches host-side polling
# off, so the two never poll at once.
#
# Built in but inert: it loads only if the host sets KVM_HINTS_REALTIME or the
# command line forces it. The hint is the wrong tool -- it also promises the
# vCPUs are never preempted, and the guest drops paravirt spinlocks on the
# strength of that, which a shared set of host cores cannot promise. Forcing
# the idle driver gets the polling and nothing else. Built into the command
# line, which is prepended to what the host passes, so no caller has to know.
# The cost is host CPU spent spinning on an idle vCPU.
#
# Prepended, not replacing: with OVERRIDE off the host's arguments come after
# this and the last value wins, so `cpuidle_haltpoll.force=0` from the host
# still turns it off.
CONFIG_HALTPOLL_CPUIDLE=y
CONFIG_CPU_IDLE_GOV_HALTPOLL=y
CONFIG_CMDLINE_BOOL=y
CONFIG_CMDLINE="cpuidle_haltpoll.force=1 swiotlb=noforce"
# CONFIG_CMDLINE_OVERRIDE is not set
# ── No bounce buffer ─────────────────────────────────────
# With more than about 3 GiB of RAM the guest has memory above 4 GiB, and x86
# then sets aside a 64 MiB SWIOTLB bounce buffer and zeroes it at boot, which
# makes the host back all of it. Nothing here ever bounces: the virtio devices
# do not offer ACCESS_PLATFORM, so virtio skips the DMA API entirely, and there
# is no other DMA-capable device. `swiotlb=noforce` in the command line above
# skips the allocation. A device that genuinely needs a 32-bit DMA mask would
# fail to map rather than bounce; there is none.
# ── microVM transport ────────────────────────────────────
CONFIG_VIRTIO=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_BLK=y # root on /dev/vda
CONFIG_VIRTIO_CONSOLE=y # hvc0, the only way into a guest that will not boot
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_FS=y # the shares the boot descriptor names
CONFIG_FUSE_FS=y # virtiofs needs it
CONFIG_VSOCKETS=y # neshub's link to the host
CONFIG_VIRTIO_VSOCKETS=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_DRM=y
CONFIG_DRM_VIRTIO_GPU=y
# ── Filesystems ──────────────────────────────────────────
CONFIG_EXT4_FS=y # output/rootfs.ext4, and a box's writable install layer
CONFIG_MISC_FILESYSTEMS=y # the menu EROFS lives under; off in base.config, which drops it silently
CONFIG_EROFS_FS=y # a game's build image, the install overlay's lower layer
CONFIG_OVERLAY_FS=y # the session overlay over a read-only install
# ── Memory typing ────────────────────────────────────────
# Page Attribute Table, and the MTRR support it depends on.
#
# Both are `def_bool y` upstream, promptable only under EXPERT -- so they were
# switched off deliberately at some point, and the guest booted logging
# "PAT support disabled because CONFIG_X86_PAT is disabled in the kernel".
#
# Without PAT there are no write-combining mappings. This guest exists to push
# frames through virtio-gpu, and WC on the GPU aperture is precisely what PAT
# provides; every frame would go through uncached or writeback mappings
# instead. Not a correctness problem, which is why it went unnoticed.
CONFIG_MTRR=y
CONFIG_X86_PAT=y
# ── POSIX ACLs ───────────────────────────────────────────
# Nothing depends on the ACLs themselves; without them, anything that sets one
# on a device node or a file logs "Operation not supported" at every boot, and
# a boot log full of failures that do not matter hides the ones that do.
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_TMPFS_POSIX_ACL=y
# ── Performance ──────────────────────────────────────────
# Chosen, not required. The mainline subset of what CachyOS enables, minus
# anything needing patches. PREEMPT_LAZY is the tree's own default and is left
# alone; PREEMPT_DYNAMIC on top allows `preempt=full` from the command line
# without a rebuild.
CONFIG_PREEMPT_DYNAMIC=y
CONFIG_LRU_GEN=y
CONFIG_LRU_GEN_ENABLED=y
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
CONFIG_PSI=y
# ── Undoing the minimal seed ─────────────────────────────
# base.config descends from a size-minimised config with EXPERT on, and EXPERT
# is what makes each of these promptable. Every one is `default y` (or the
# default choice) upstream, and every one was off in the built kernel. They
# are pinned here because nothing else would stop a reseed from bringing them
# back.
#
# SLUB_TINY drops the per-CPU slab caches, so every kmalloc and kfree takes the
# locked slow path. Its own help text says it is not for performance. This
# guest allocates constantly: network buffers, GPU ioctls, io_uring, futexes.
# CONFIG_SLUB_TINY is not set
#
# Without ADVISE_SYSCALLS, madvise() and fadvise() return ENOSYS. THP above is
# madvise-only, so no user mapping could ever get a huge page, and allocators
# could never hand memory back with MADV_DONTNEED or MADV_FREE.
CONFIG_ADVISE_SYSCALLS=y
#
# Without MEMBARRIER, Wine and the .NET runtime fall back to an mprotect() trick
# to flush other threads' write buffers, which forces a TLB shootdown across
# every vCPU -- an IPI storm, and each IPI is a VM exit. RSEQ gives glibc and
# per-CPU allocators their fast paths.
CONFIG_MEMBARRIER=y
CONFIG_RSEQ=y
#
# -O2, not -Os: the whole kernel was being compiled for size, and a guest that
# runs Wine spends much of its time in syscalls.
CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
#
# Without jump labels, every static key is a memory load and a branch instead
# of a patched no-op, on paths such as steal-time accounting, the scheduler and
# cgroup checks.
CONFIG_JUMP_LABEL=y
# Autogroup is deliberately OFF, which is a departure from CachyOS.
#
# It groups tasks by session id, which is a desktop tuning: it keeps a
# terminal's `make -j` from starving the browser. This guest is a
# single-application appliance with no interactive sessions to balance
# against each other, so there is nothing for it to do but add overhead.
# CONFIG_SCHED_AUTOGROUP is not set
# ── Deliberately absent ──────────────────────────────────
# No CONFIG_SND of any kind. The guest has no sound hardware and wants none:
# neswire is a virtual PipeWire sink, and an ALSA stack would only add a second
# thing that could claim to be the default output.
# Speculation mitigations are off, as a decision rather than an oversight.
#
# `lscpu` in the guest reports Spectre v1/v2, SSB, SRSO and TSA as Vulnerable,
# and that is intended. This is a single-tenant sandbox: one player's session,
# torn down at the end of it, with the VM boundary as the isolation. The
# attacks these defend against are cross-tenant side channels, and the cost is
# paid on every syscall and context switch in a workload that is latency
# bound.
#
# What would change this: running two players' sessions on one box at the same
# time, or anything of value living inside the guest. Neither is true today.
# CONFIG_CPU_MITIGATIONS is not set
build/patches/mesa/0003-radv-amdgpu-cache-AMDGPU_INFO_MEMORY-briefly-on-the-.patch
-140
View File
@@ -1,140 +0,0 @@
From 4dcb0baf29136c90776133afb714bac3c4cb7686 Mon Sep 17 00:00:00 2001
From: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Date: Thu, 17 Sep 2026 21:50:25 +0300
Subject: [PATCH] radv/amdgpu: cache AMDGPU_INFO_MEMORY briefly on the virtio
path
Heap usage is asked for far more often than it changes. Measured on an
amdgpu native context with one game running, RADV asked for it ~47 times
per 8.8 ms frame -- the same answer, 47 times, each one a synchronous round
trip to the host rather than an ioctl.
Serve a recent answer instead, for one millisecond. That is inside the
contract of what the value is for: heap usage feeds VK_EXT_memory_budget
and the winsys's own eviction decisions, and the spec calls those estimates
that may be out of date. It is also short enough that an application
allocating hard still sees its own pressure within a frame at any plausible
frame rate.
Only on the virtio path, where a query costs a round trip; a local ioctl is
cheap enough that caching it would be complexity for nothing. Only this
query, and only at its natural size: every other AMDGPU_INFO_* either
carries a caller-supplied selector in the union, so one cached answer would
be the wrong answer to the next question, or is asked once at startup.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
src/amd/common/virtio/amdgpu_virtio.c | 53 +++++++++++++++++++
src/amd/common/virtio/amdgpu_virtio_device.c | 1 +
src/amd/common/virtio/amdgpu_virtio_private.h | 6 +++
3 files changed, 60 insertions(+)
diff --git a/src/amd/common/virtio/amdgpu_virtio.c b/src/amd/common/virtio/amdgpu_virtio.c
index 00494d6a750..dd38ef141f5 100644
--- a/src/amd/common/virtio/amdgpu_virtio.c
+++ b/src/amd/common/virtio/amdgpu_virtio.c
@@ -19,8 +19,25 @@
#include "drm-uapi/amdgpu_drm.h"
#include "util/log.h"
+#include "util/os_time.h"
#include "util/u_math.h"
+/* How long a cached AMDGPU_INFO_MEMORY answer is reused, in nanoseconds.
+ *
+ * Bounded by what the value is for. Heap usage feeds VK_EXT_memory_budget and
+ * the winsys's own eviction decisions, and the spec calls those values
+ * estimates that may be out of date -- so a millisecond of staleness is inside
+ * the contract, while a round trip per ask is not free here the way it is on a
+ * local ioctl.
+ *
+ * A millisecond rather than a frame: it is short enough that an application
+ * allocating hard still sees its own pressure within a frame at any plausible
+ * rate, and long enough to collapse the repeats. Measured on an amdgpu native
+ * context with one game running, RADV asked for this ~47 times per 8.8 ms
+ * frame -- the same answer, 47 synchronous round trips.
+ */
+#define AMDVGPU_MEMORY_INFO_TTL_NS (1000 * 1000)
+
int
amdvgpu_query_info(amdvgpu_device_handle dev, struct drm_amdgpu_info *info)
{
@@ -32,6 +49,30 @@ amdvgpu_query_info(amdvgpu_device_handle dev, struct drm_amdgpu_info *info)
struct amdgpu_ccmd_query_info_rsp *rsp;
assert(0 == (offsetof(struct amdgpu_ccmd_query_info_rsp, payload) % 8));
+ /* AMDGPU_INFO_MEMORY is asked far more often than it changes, and over
+ * virtio every ask is a synchronous round trip to the host rather than an
+ * ioctl. Serve a recent answer instead.
+ *
+ * Only this query, and only at its natural size: every other query either
+ * has a caller-supplied selector in the union -- so one cached answer would
+ * be the wrong answer to the next question -- or is asked once at startup
+ * and costs nothing.
+ */
+ bool cacheable = info->query == AMDGPU_INFO_MEMORY &&
+ info->return_size == sizeof(struct drm_amdgpu_memory_info);
+ if (cacheable) {
+ int64_t now = os_time_get_nano();
+ simple_mtx_lock(&dev->memory_info_mutex);
+ if (dev->memory_info_stamp &&
+ now - dev->memory_info_stamp < AMDVGPU_MEMORY_INFO_TTL_NS) {
+ memcpy((void *)(uintptr_t)info->return_pointer, &dev->memory_info,
+ sizeof(dev->memory_info));
+ simple_mtx_unlock(&dev->memory_info_mutex);
+ return 0;
+ }
+ simple_mtx_unlock(&dev->memory_info_mutex);
+ }
+
req->hdr = AMDGPU_CCMD(QUERY_INFO, req_len);
memcpy(&req->info, info, sizeof(struct drm_amdgpu_info));
@@ -43,6 +84,18 @@ amdvgpu_query_info(amdvgpu_device_handle dev, struct drm_amdgpu_info *info)
memcpy((void*)(uintptr_t)info->return_pointer, rsp->payload, info->return_size);
+ if (cacheable) {
+ simple_mtx_lock(&dev->memory_info_mutex);
+ memcpy(&dev->memory_info, rsp->payload, sizeof(dev->memory_info));
+ /* Stamped after the answer is in hand, so the window covers the time the
+ * value is actually served rather than the round trip that fetched it.
+ * A zero stamp means "never fetched", so a clock that returns zero here
+ * costs a re-fetch rather than pinning a stale answer forever.
+ */
+ dev->memory_info_stamp = os_time_get_nano();
+ simple_mtx_unlock(&dev->memory_info_mutex);
+ }
+
return 0;
}
diff --git a/src/amd/common/virtio/amdgpu_virtio_device.c b/src/amd/common/virtio/amdgpu_virtio_device.c
index eecfd6aa11a..7e58913d520 100644
--- a/src/amd/common/virtio/amdgpu_virtio_device.c
+++ b/src/amd/common/virtio/amdgpu_virtio_device.c
@@ -136,6 +136,7 @@ int amdvgpu_device_initialize(int fd, uint32_t *drm_major, uint32_t *drm_minor,
dev->vdev = vdev;
simple_mtx_init(&dev->handle_to_vbo_mutex, mtx_plain);
+ simple_mtx_init(&dev->memory_info_mutex, mtx_plain);
simple_mtx_init(&dev->contexts_mutex, mtx_plain);
dev->handle_to_vbo = _mesa_hash_table_u64_create(NULL);
diff --git a/src/amd/common/virtio/amdgpu_virtio_private.h b/src/amd/common/virtio/amdgpu_virtio_private.h
index 743f8f74b5c..fb877dc8d28 100644
--- a/src/amd/common/virtio/amdgpu_virtio_private.h
+++ b/src/amd/common/virtio/amdgpu_virtio_private.h
@@ -60,6 +60,12 @@ struct amdvgpu_device {
struct drm_amdgpu_info_device dev_info;
+ /* AMDGPU_INFO_MEMORY, cached for a short while. See amdvgpu_query_info(). */
+ simple_mtx_t memory_info_mutex;
+ struct drm_amdgpu_memory_info memory_info;
+ /* os_time_get_nano() when memory_info was fetched; 0 means never. */
+ int64_t memory_info_stamp;
+
/* Blob id are per drm_file identifiers of host blobs.
* Use a monotically increased integer to assign the blob id.
*/
build/patches/mesa/0003-virtio-vdrm-ac-log-where-threads-wait-per-wait-and-p.patch
+602
View File
@@ -0,0 +1,602 @@
From 35db2e58006deca5c6383517f9228eb32a608548 Mon Sep 17 00:00:00 2001
From: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Date: Wed, 23 Sep 2026 20:13:47 +0300
Subject: [PATCH] virtio/vdrm, ac: log where threads wait, per wait and per
second
A guest thread under a native context can be blocked on the host in more
places than it looks: a synchronous request, a submission that holds
eb_lock across an ioctl which itself waits on in-fences, a BO wait, or an
ordinary syncobj wait. When a frame runs long there is no way to tell from
the outside which of those it was, or whether it was any of them.
Two knobs, in milliseconds, both off when unset or 0:
- MESA_SLOW_WAIT_MS logs each single wait longer than the value, with its
command, ring and how the time split between waiting for eb_lock, the
submit or flush ioctl, the fence, and the host catching up.
- MESA_WAIT_STATS logs, once a second per thread, the time and count spent
in each kind of wait, when the total reached the value. Many short waits
that add up to a long frame never trip a per-wait threshold; totals show
them, and a second with a long frame can be compared with one without.
Syncobj waits are timed by wrapping the device's sync provider, since the
Vulkan runtime waits through it directly and never through the ac_drm_cs_*
helpers. The wrapper is installed only when a knob is set, and times only
waits that could block; a poll is not.
Every line carries a UTC wall-clock stamp and the thread id, so it can be
lined up with other components' logs. The shared pieces live in a header-only
util/u_wait_log.h, so no build file changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
src/amd/common/ac_linux_drm.c | 197 +++++++++++++++++++++++++++++++++
src/util/u_wait_log.h | 133 ++++++++++++++++++++++
src/virtio/vdrm/vdrm.c | 80 +++++++++++++
src/virtio/vdrm/vdrm.h | 14 +++
src/virtio/vdrm/vdrm_virtgpu.c | 20 +++-
5 files changed, 443 insertions(+), 1 deletion(-)
create mode 100644 src/util/u_wait_log.h
diff --git a/src/amd/common/ac_linux_drm.c b/src/amd/common/ac_linux_drm.c
index 63b27058ec1..d2f16aefc83 100644
--- a/src/amd/common/ac_linux_drm.c
+++ b/src/amd/common/ac_linux_drm.c
@@ -14,10 +14,205 @@
#include <time.h>
#include <unistd.h>
+#include "util/u_wait_log.h"
+
#ifdef HAVE_AMDGPU_VIRTIO
#include "virtio/amdgpu_virtio.h"
#endif
+/* Wait logging for blocking syncobj waits -- what vkWaitForFences and timeline
+ * semaphore waits come down to. See util/u_wait_log.h for the knobs; they are
+ * the virtio transport's too, so one setting shows both a thread waiting on
+ * the GPU and a thread waiting on the host. Polls (a zero timeout) are never
+ * timed.
+ *
+ * Done by wrapping the device's sync provider rather than the ac_drm_cs_*
+ * helpers, because the Vulkan runtime waits through the provider directly and
+ * never calls those helpers: timing them saw almost nothing. The wrapper is
+ * installed only when a knob is set, so it costs nothing otherwise.
+ */
+enum { SYNCOBJ_WAIT, TIMELINE_WAIT, WAIT_KINDS };
+static const char *const wait_kinds[WAIT_KINDS] = { "syncobj", "timeline" };
+static __thread struct u_wait_log_window wait_window;
+
+static void
+note_wait(unsigned kind, int64_t t0, unsigned num_handles, int ret)
+{
+ int64_t took = os_time_get_nano() - t0;
+ u_wait_log_account(&wait_window, "sync", wait_kinds, WAIT_KINDS, kind, took);
+
+ int64_t slow = u_wait_log_slow_ns();
+ if (slow && took > slow) {
+ char stamp[16];
+ u_wait_log_stamp(stamp);
+ mesa_logw("%s sync: %s wait on %u syncobj(s) took %.1f ms (ret %d) tid %d", stamp,
+ wait_kinds[kind], num_handles, took / 1e6, ret, gettid());
+ }
+}
+
+struct timed_sync_provider {
+ struct util_sync_provider base;
+ struct util_sync_provider *inner;
+};
+
+static struct util_sync_provider *
+inner_of(struct util_sync_provider *p)
+{
+ return ((struct timed_sync_provider *)p)->inner;
+}
+
+static int
+timed_create(struct util_sync_provider *p, uint32_t flags, uint32_t *handle)
+{
+ return inner_of(p)->create(inner_of(p), flags, handle);
+}
+
+static int
+timed_destroy(struct util_sync_provider *p, uint32_t handle)
+{
+ return inner_of(p)->destroy(inner_of(p), handle);
+}
+
+static int
+timed_handle_to_fd(struct util_sync_provider *p, uint32_t handle, int *out_obj_fd)
+{
+ return inner_of(p)->handle_to_fd(inner_of(p), handle, out_obj_fd);
+}
+
+static int
+timed_fd_to_handle(struct util_sync_provider *p, int obj_fd, uint32_t *handle)
+{
+ return inner_of(p)->fd_to_handle(inner_of(p), obj_fd, handle);
+}
+
+static int
+timed_import_sync_file(struct util_sync_provider *p, uint32_t handle, int sync_file_fd)
+{
+ return inner_of(p)->import_sync_file(inner_of(p), handle, sync_file_fd);
+}
+
+static int
+timed_export_sync_file(struct util_sync_provider *p, uint32_t handle, int *out_sync_file_fd)
+{
+ return inner_of(p)->export_sync_file(inner_of(p), handle, out_sync_file_fd);
+}
+
+static int
+timed_wait(struct util_sync_provider *p, uint32_t *handles, unsigned num_handles,
+ int64_t timeout_nsec, unsigned flags, uint32_t *first_signaled)
+{
+ struct util_sync_provider *in = inner_of(p);
+ if (!timeout_nsec)
+ return in->wait(in, handles, num_handles, timeout_nsec, flags, first_signaled);
+
+ int64_t t0 = os_time_get_nano();
+ int ret = in->wait(in, handles, num_handles, timeout_nsec, flags, first_signaled);
+ note_wait(SYNCOBJ_WAIT, t0, num_handles, ret);
+ return ret;
+}
+
+static int
+timed_reset(struct util_sync_provider *p, const uint32_t *handles, uint32_t handle_count)
+{
+ return inner_of(p)->reset(inner_of(p), handles, handle_count);
+}
+
+static int
+timed_signal(struct util_sync_provider *p, const uint32_t *handles, uint32_t handle_count)
+{
+ return inner_of(p)->signal(inner_of(p), handles, handle_count);
+}
+
+static int
+timed_timeline_signal(struct util_sync_provider *p, const uint32_t *handles, uint64_t *points,
+ uint32_t handle_count)
+{
+ return inner_of(p)->timeline_signal(inner_of(p), handles, points, handle_count);
+}
+
+static int
+timed_timeline_wait(struct util_sync_provider *p, uint32_t *handles, uint64_t *points,
+ unsigned num_handles, int64_t timeout_nsec, unsigned flags,
+ uint32_t *first_signaled)
+{
+ struct util_sync_provider *in = inner_of(p);
+ if (!timeout_nsec)
+ return in->timeline_wait(in, handles, points, num_handles, timeout_nsec, flags,
+ first_signaled);
+
+ int64_t t0 = os_time_get_nano();
+ int ret = in->timeline_wait(in, handles, points, num_handles, timeout_nsec, flags,
+ first_signaled);
+ note_wait(TIMELINE_WAIT, t0, num_handles, ret);
+ return ret;
+}
+
+static int
+timed_query(struct util_sync_provider *p, uint32_t *handles, uint64_t *points,
+ uint32_t handle_count, uint32_t flags)
+{
+ return inner_of(p)->query(inner_of(p), handles, points, handle_count, flags);
+}
+
+static int
+timed_transfer(struct util_sync_provider *p, uint32_t dst_handle, uint64_t dst_point,
+ uint32_t src_handle, uint64_t src_point, uint32_t flags)
+{
+ return inner_of(p)->transfer(inner_of(p), dst_handle, dst_point, src_handle, src_point,
+ flags);
+}
+
+static struct util_sync_provider *wrap_timed(struct util_sync_provider *inner);
+
+static void
+timed_finalize(struct util_sync_provider *p)
+{
+ inner_of(p)->finalize(inner_of(p));
+ free(p);
+}
+
+static struct util_sync_provider *
+timed_clone(struct util_sync_provider *p)
+{
+ struct util_sync_provider *cloned = inner_of(p)->clone(inner_of(p));
+ return cloned ? wrap_timed(cloned) : NULL;
+}
+
+/* Wrap `inner` so its blocking waits are timed. On allocation failure the
+ * inner provider is returned as it is: logging is not worth failing a device.
+ */
+static struct util_sync_provider *
+wrap_timed(struct util_sync_provider *inner)
+{
+ struct timed_sync_provider *t = calloc(1, sizeof(*t));
+ if (!t)
+ return inner;
+ t->inner = inner;
+ /* An operation the inner provider leaves NULL stays NULL here, since
+ * callers test for it rather than call through it.
+ */
+#define FWD(op) .op = inner->op ? timed_##op : NULL
+ t->base = (struct util_sync_provider) {
+ FWD(create),
+ FWD(destroy),
+ FWD(handle_to_fd),
+ FWD(fd_to_handle),
+ FWD(import_sync_file),
+ FWD(export_sync_file),
+ FWD(wait),
+ FWD(reset),
+ FWD(signal),
+ FWD(timeline_signal),
+ FWD(timeline_wait),
+ FWD(query),
+ FWD(transfer),
+ FWD(finalize),
+ FWD(clone),
+ };
+#undef FWD
+ return &t->base;
+}
+
struct ac_drm_device {
union {
amdgpu_device_handle adev;
@@ -82,6 +277,8 @@ int ac_drm_device_initialize(int fd, bool is_virtio,
}
if (r == 0) {
+ if (u_wait_log_enabled() && (*dev)->p)
+ (*dev)->p = wrap_timed((*dev)->p);
(*dev)->is_virtio = is_virtio;
/* Device-static, so it is asked once here rather than on every heap
* query. A failure is not fatal: the caller falls back to querying it.
diff --git a/src/util/u_wait_log.h b/src/util/u_wait_log.h
new file mode 100644
index 00000000000..ce96256f371
--- /dev/null
+++ b/src/util/u_wait_log.h
@@ -0,0 +1,133 @@
+/*
+ * SPDX-License-Identifier: MIT
+ */
+
+/* Logging for time a thread spends waiting, for finding what a frame that
+ * runs long was blocked on.
+ *
+ * Two knobs, both in milliseconds and both off when unset or 0:
+ *
+ * - MESA_SLOW_WAIT_MS: log each single wait longer than this.
+ * - MESA_WAIT_STATS: once a second, per thread, log how much time went to
+ * each kind of wait, if the total reached this. Many
+ * short waits that add up to a long frame never trip a
+ * per-wait threshold; totals show them.
+ *
+ * Every line carries a UTC wall-clock stamp, so it can be lined up with other
+ * components' logs. Header-only, so each file that uses it keeps its own
+ * per-thread window and reports under its own name.
+ */
+
+#ifndef U_WAIT_LOG_H
+#define U_WAIT_LOG_H
+
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "util/log.h"
+#include "util/os_time.h"
+#include "util/u_debug.h"
+
+#define U_WAIT_LOG_MAX_KINDS 8
+
+struct u_wait_log_window {
+ int64_t start;
+ int64_t ns[U_WAIT_LOG_MAX_KINDS];
+ uint32_t n[U_WAIT_LOG_MAX_KINDS];
+};
+
+/* A millisecond option as nanoseconds, read once into *cache. */
+static inline int64_t
+u_wait_log_option_ns(const char *name, int64_t *cache)
+{
+ if (*cache < 0)
+ *cache = debug_get_num_option(name, 0) * 1000000ll;
+ return *cache;
+}
+
+static inline int64_t
+u_wait_log_slow_ns(void)
+{
+ static int64_t cache = -1;
+ return u_wait_log_option_ns("MESA_SLOW_WAIT_MS", &cache);
+}
+
+static inline int64_t
+u_wait_log_stats_ns(void)
+{
+ static int64_t cache = -1;
+ return u_wait_log_option_ns("MESA_WAIT_STATS", &cache);
+}
+
+/* Whether any wait timing is wanted at all. */
+static inline bool
+u_wait_log_enabled(void)
+{
+ return u_wait_log_slow_ns() || u_wait_log_stats_ns();
+}
+
+/* "HH:MM:SS.mmm" in UTC. */
+static inline void
+u_wait_log_stamp(char buf[16])
+{
+ struct timespec ts;
+ struct tm tm;
+ clock_gettime(CLOCK_REALTIME, &ts);
+ gmtime_r(&ts.tv_sec, &tm);
+ snprintf(buf, 16, "%02d:%02d:%02d.%03ld", tm.tm_hour, tm.tm_min, tm.tm_sec,
+ ts.tv_nsec / 1000000);
+}
+
+/* Add one wait of `ns` nanoseconds of kind `kind` to this thread's window,
+ * and report and reset the window once it is a second old.
+ */
+static inline void
+u_wait_log_account(struct u_wait_log_window *w, const char *who,
+ const char *const *kinds, unsigned num_kinds,
+ unsigned kind, int64_t ns)
+{
+ int64_t floor = u_wait_log_stats_ns();
+ if (!floor || kind >= num_kinds || num_kinds > U_WAIT_LOG_MAX_KINDS)
+ return;
+
+ int64_t now = os_time_get_nano();
+ if (!w->start)
+ w->start = now;
+ w->ns[kind] += ns;
+ w->n[kind]++;
+
+ if (now - w->start < 1000000000ll)
+ return;
+
+ int64_t total = 0;
+ for (unsigned i = 0; i < num_kinds; i++)
+ total += w->ns[i];
+
+ if (total >= floor) {
+ char line[256];
+ size_t len = 0;
+ line[0] = '\0';
+ for (unsigned i = 0; i < num_kinds && len < sizeof(line); i++) {
+ if (!w->n[i])
+ continue;
+ int wrote = snprintf(line + len, sizeof(line) - len, " %s %.1f/%u",
+ kinds[i], w->ns[i] / 1e6, w->n[i]);
+ if (wrote < 0)
+ break;
+ len += wrote;
+ }
+ char stamp[16];
+ u_wait_log_stamp(stamp);
+ mesa_logw("%s %s: tid %d waited %.1f ms of %.0f:%s", stamp, who, gettid(),
+ total / 1e6, (now - w->start) / 1e6, line);
+ }
+
+ memset(w, 0, sizeof(*w));
+ w->start = now;
+}
+
+#endif /* U_WAIT_LOG_H */
diff --git a/src/virtio/vdrm/vdrm.c b/src/virtio/vdrm/vdrm.c
index d09df9bb99d..4819219cbc5 100644
--- a/src/virtio/vdrm/vdrm.c
+++ b/src/virtio/vdrm/vdrm.c
@@ -4,10 +4,40 @@
*/
#include "util/u_math.h"
+#include "util/u_wait_log.h"
#include "util/perf/cpu_trace.h"
#include "vdrm.h"
+/* Wait logging for the transport: see util/u_wait_log.h for the knobs.
+ *
+ * Every path here can stall the caller on the host, and execbuf holds eb_lock
+ * while it does, which stalls every other thread of this device too. The
+ * split into lock, flush, fence, host and submit time says which.
+ */
+static const char *const wait_kinds[VDRM_WAIT_COUNT] = {
+ [VDRM_WAIT_LOCK] = "lock",
+ [VDRM_WAIT_FLUSH] = "flush",
+ [VDRM_WAIT_FENCE] = "fence",
+ [VDRM_WAIT_HOST] = "host",
+ [VDRM_WAIT_SUBMIT] = "submit",
+ [VDRM_WAIT_BO] = "bo_wait",
+};
+
+static __thread struct u_wait_log_window wait_window;
+
+void
+vdrm_account_wait(enum vdrm_wait kind, int64_t ns)
+{
+ u_wait_log_account(&wait_window, "vdrm", wait_kinds, VDRM_WAIT_COUNT, kind, ns);
+}
+
+static double
+ms(int64_t ns)
+{
+ return ns / 1e6;
+}
+
struct vdrm_device * vdrm_virtgpu_connect(int fd, uint32_t context_type);
struct vdrm_device * vdrm_vpipe_connect(uint32_t context_type);
@@ -114,7 +144,11 @@ vdrm_execbuf(struct vdrm_device *vdev, struct vdrm_execbuf_params *p)
MESA_TRACE_FUNC();
+ bool timed = u_wait_log_enabled();
+ int64_t t0 = timed ? os_time_get_nano() : 0;
+
simple_mtx_lock(&vdev->eb_lock);
+ int64_t t_locked = timed ? os_time_get_nano() : 0;
p->req->seqno = ++vdev->next_seqno;
@@ -127,6 +161,23 @@ vdrm_execbuf(struct vdrm_device *vdev, struct vdrm_execbuf_params *p)
out_unlock:
simple_mtx_unlock(&vdev->eb_lock);
+ if (timed) {
+ int64_t t_done = os_time_get_nano();
+ vdrm_account_wait(VDRM_WAIT_LOCK, t_locked - t0);
+ vdrm_account_wait(VDRM_WAIT_SUBMIT, t_done - t_locked);
+
+ int64_t slow = u_wait_log_slow_ns();
+ if (slow && t_done - t0 > slow) {
+ char stamp[16];
+ u_wait_log_stamp(stamp);
+ mesa_logw("%s vdrm: execbuf cmd %u ring %d took %.1f ms (lock %.1f, submit %.1f; "
+ "%u in-syncobjs, in-fence %s) tid %d",
+ stamp, p->req->cmd, p->ring_idx, ms(t_done - t0), ms(t_locked - t0),
+ ms(t_done - t_locked), p->num_in_syncobjs,
+ p->has_in_fence_fd ? "yes" : "no", gettid());
+ }
+ }
+
return ret;
}
@@ -141,7 +192,11 @@ vdrm_send_req(struct vdrm_device *vdev, struct vdrm_ccmd_req *req, bool sync)
uintptr_t fence = 0;
int ret = 0;
+ bool timed = u_wait_log_enabled();
+ int64_t t0 = timed ? os_time_get_nano() : 0;
+
simple_mtx_lock(&vdev->eb_lock);
+ int64_t t_locked = timed ? os_time_get_nano() : 0;
ret = enqueue_req(vdev, req);
if (ret || !sync)
@@ -151,16 +206,41 @@ vdrm_send_req(struct vdrm_device *vdev, struct vdrm_ccmd_req *req, bool sync)
out_unlock:
simple_mtx_unlock(&vdev->eb_lock);
+ int64_t t_flushed = timed ? os_time_get_nano() : 0;
if (ret)
return ret;
+ int64_t t_fenced = t_flushed;
if (sync) {
MESA_TRACE_SCOPE("vdrm_execbuf sync");
vdev->funcs->wait_fence(vdev, fence);
+ if (timed)
+ t_fenced = os_time_get_nano();
vdrm_host_sync(vdev, req);
}
+ if (timed) {
+ int64_t t_done = os_time_get_nano();
+ vdrm_account_wait(VDRM_WAIT_LOCK, t_locked - t0);
+ vdrm_account_wait(VDRM_WAIT_FLUSH, t_flushed - t_locked);
+ if (sync) {
+ vdrm_account_wait(VDRM_WAIT_FENCE, t_fenced - t_flushed);
+ vdrm_account_wait(VDRM_WAIT_HOST, t_done - t_fenced);
+ }
+
+ int64_t slow = u_wait_log_slow_ns();
+ if (slow && t_done - t0 > slow) {
+ char stamp[16];
+ u_wait_log_stamp(stamp);
+ mesa_logw("%s vdrm: %s ccmd %u took %.1f ms (lock %.1f, flush %.1f, "
+ "fence %.1f, host %.1f) tid %d",
+ stamp, sync ? "sync" : "async", req->cmd, ms(t_done - t0),
+ ms(t_locked - t0), ms(t_flushed - t_locked),
+ ms(t_fenced - t_flushed), ms(t_done - t_fenced), gettid());
+ }
+ }
+
return 0;
}
diff --git a/src/virtio/vdrm/vdrm.h b/src/virtio/vdrm/vdrm.h
index fa676896f23..dbb6bfd2441 100644
--- a/src/virtio/vdrm/vdrm.h
+++ b/src/virtio/vdrm/vdrm.h
@@ -114,6 +114,20 @@ int vdrm_execbuf(struct vdrm_device *vdev, struct vdrm_execbuf_params *p);
void vdrm_host_sync(struct vdrm_device *vdev, const struct vdrm_ccmd_req *req);
+/* Kinds of wait the transport accounts for; see util/u_wait_log.h. */
+enum vdrm_wait {
+ VDRM_WAIT_LOCK, /* waiting for eb_lock */
+ VDRM_WAIT_FLUSH, /* the flush ioctl, lock held */
+ VDRM_WAIT_FENCE, /* a sync request's fence */
+ VDRM_WAIT_HOST, /* the host catching up after the fence */
+ VDRM_WAIT_SUBMIT, /* the execbuf ioctl, lock held */
+ VDRM_WAIT_BO, /* a BO wait */
+ VDRM_WAIT_COUNT,
+};
+
+/* Add one wait to this thread's per-second totals (MESA_WAIT_STATS). */
+void vdrm_account_wait(enum vdrm_wait kind, int64_t ns);
+
/**
* Import dmabuf fd returning a GEM handle
*/
diff --git a/src/virtio/vdrm/vdrm_virtgpu.c b/src/virtio/vdrm/vdrm_virtgpu.c
index bccd4a5d279..9b9d5dc63c4 100644
--- a/src/virtio/vdrm/vdrm_virtgpu.c
+++ b/src/virtio/vdrm/vdrm_virtgpu.c
@@ -13,6 +13,7 @@
#include "drm-uapi/virtgpu_drm.h"
#include "util/libsync.h"
#include "util/log.h"
+#include "util/u_wait_log.h"
#include "util/perf/cpu_trace.h"
@@ -213,9 +214,26 @@ virtgpu_bo_wait(struct vdrm_device *vdev, uint32_t handle)
};
int ret;
+ bool timed = u_wait_log_enabled();
+ int64_t t0 = timed ? os_time_get_nano() : 0;
+
/* Side note, this ioctl is defined as IO_WR but should be IO_W: */
ret = virtgpu_ioctl(vgdev->fd, VIRTGPU_WAIT, &args);
- if (ret && errno == EBUSY)
+ int err = errno;
+
+ if (timed) {
+ int64_t took = os_time_get_nano() - t0;
+ vdrm_account_wait(VDRM_WAIT_BO, took);
+ int64_t slow = u_wait_log_slow_ns();
+ if (slow && took > slow) {
+ char stamp[16];
+ u_wait_log_stamp(stamp);
+ mesa_logw("%s vdrm: bo_wait handle %u took %.1f ms tid %d", stamp, handle,
+ took / 1e6, gettid());
+ }
+ }
+
+ if (ret && err == EBUSY)
return -EBUSY;
return 0;
--
2.55.0
build/patches/proton-ge/0001-kaldi-order-i386-steps-only-when-they-exist.patch
+26
View File
@@ -0,0 +1,26 @@
kaldi's i386 and x86_64 builds share one source directory, and its cmake
writes generated files into it, so proton-ge serializes the two with plain
ordering rules. Those rules name the i386 steps unconditionally.
A wow64-only build has no i386-unix in ARCHS, so the i386 steps are never
defined, and make stops with "no rule to make target .kaldi-i386-configure".
Only order against them when they exist.
diff --git a/Makefile.in b/Makefile.in
index f8d959c..9b92fec 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -573,9 +573,13 @@ $(eval $(call rules-cmake,kaldi,aarch64,unix))
# the same source directory.
#
# The below assures that only a single configure/build step can run at a time.
+# The i386 steps exist only when i386-unix is in ARCHS; a wow64-only build has
+# none, and ordering against them would ask for a target with no rule.
+ifneq ($(findstring i386-unix,$(ARCHS)),)
$(OBJ)/.kaldi-x86_64-configure: $(OBJ)/.kaldi-i386-configure
$(OBJ)/.kaldi-i386-build: $(OBJ)/.kaldi-x86_64-configure
$(OBJ)/.kaldi-i386-build: $(OBJ)/.kaldi-x86_64-build
+endif
##
## vosk
build/patches/proton-ge/0002-protonfixes-fetch-unzip-from-snapshot.debian.org.patch
+22
View File
@@ -0,0 +1,22 @@
protonfixes pins unzip to Debian revision 6.0-29 and fetches it from the
live pool. The pool keeps only current revisions, so once 6.0-31 landed the
download started failing with a 404 and the build with it.
snapshot.debian.org keeps every revision Debian has published, at a stable
path per timestamp. This is the first snapshot that carries 6.0-29, and the
orig tarball has been in the pool since 2009, so both files resolve there
and are the same bytes the pin always meant.
diff --git a/protonfixes/Makefile b/protonfixes/Makefile
index 4dcd851..8fab279 100644
--- a/protonfixes/Makefile
+++ b/protonfixes/Makefile
@@ -146,7 +146,7 @@ DEFINES = -DACORN_FTYPE_NFS -DWILD_STOP_AT_DIR -DLARGE_FILE_SUPPORT \
-DNO_WORKING_ISPRINT
UNZIP_VERSION := 6.0
UNZIP_DEBIAN_REVISION := 29
-UNZIP_BASE_URL := https://deb.debian.org/debian/pool/main/u/unzip
+UNZIP_BASE_URL := https://snapshot.debian.org/archive/debian/20250311T215724Z/pool/main/u/unzip
UNZIP_ORIG_TARBALL := unzip_$(UNZIP_VERSION).orig.tar.gz
UNZIP_DEBIAN_TARBALL := unzip_$(UNZIP_VERSION)-$(UNZIP_DEBIAN_REVISION).debian.tar.xz
UNZIP_DOWNLOAD_DIR := $(OBJDIR)/downloads/unzip
build/patches/proton-ge/0003-wine-give-each-pe-arch-its-own-vkd3d-under-wow64.patch
+43
View File
@@ -0,0 +1,43 @@
The x86_64 wine build gets VKD3D_PE_LIBS with a -L to the x86_64 vkd3d
DLLs. That is right while a separate i386 wine build exists and gets its
own value.
A wow64-only build has no i386 wine build: the x86_64 one builds the i386
PE side as well, and wine takes one VKD3D_PE_LIBS for every PE arch. The
i386 wined3d.dll then links against the x86_64 libvkd3d-1.dll and fails
with "file format not recognized". ld does not skip a mismatched DLL
named with -l:, so listing both directories does not work either.
Under wow64, keep the library names in VKD3D_PE_LIBS and pass each arch
its own directory through i386_LDFLAGS and x86_64_LDFLAGS, which wine's
configure already takes per arch. The i386 vkd3d was only ever built
as a dependency of the i386 wine build, so the x86_64 one now waits for it.
diff --git a/Makefile.in b/Makefile.in
index f8d959c..2ac0b34 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -635,9 +639,23 @@ WINE_aarch64_AUTOCONF_ARGS = \
WINE_i386_AUTOCONF_ARGS += \
VKD3D_PE_CFLAGS="-I$(VULKAN_HEADERS_i386_DST)/include -I$(VKD3D_i386_DST)/include/vkd3d" \
VKD3D_PE_LIBS="-L$(VKD3D_i386_LIBDIR)/vkd3d/i386-windows -l:libvkd3d-1.dll -l:libvkd3d-shader-1.dll -l:libvkd3d-utils-1.dll"
+ifneq ($(findstring i386-unix,$(ARCHS)),)
WINE_x86_64_AUTOCONF_ARGS += \
VKD3D_PE_CFLAGS="-I$(VULKAN_HEADERS_x86_64_DST)/include -I$(VKD3D_x86_64_DST)/include/vkd3d" \
VKD3D_PE_LIBS="-L$(VKD3D_x86_64_LIBDIR)/vkd3d/x86_64-windows -l:libvkd3d-1.dll -l:libvkd3d-shader-1.dll -l:libvkd3d-utils-1.dll"
+else
+# Without i386-unix there is no i386 wine build, and the x86_64 one builds the
+# i386 PE side too (wow64). It links both, but VKD3D_PE_LIBS is one value for
+# every PE arch, so a -L in it would hand the i386 link the x86_64 DLLs. Name
+# the libraries there and give each arch its own directory through its own
+# LDFLAGS instead.
+WINE_x86_64_AUTOCONF_ARGS += \
+ VKD3D_PE_CFLAGS="-I$(VULKAN_HEADERS_x86_64_DST)/include -I$(VKD3D_x86_64_DST)/include/vkd3d" \
+ VKD3D_PE_LIBS="-l:libvkd3d-1.dll -l:libvkd3d-shader-1.dll -l:libvkd3d-utils-1.dll"
+WINE_i386-windows_LIBFLAGS = -L$(VKD3D_i386_LIBDIR)/vkd3d/i386-windows
+WINE_x86_64-windows_LIBFLAGS = -L$(VKD3D_x86_64_LIBDIR)/vkd3d/x86_64-windows
+$(OBJ)/.wine-x86_64-build: $(OBJ)/.vkd3d-i386-post-build
+endif
WINE_DEPENDS = ffmpeg openfst kaldi vosk libxml2 libxkbcommon libpcap
WINE_i386_DEPENDS = vkd3d
build/scripts/kernel-build.sh
+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env bash
# Builds the guest kernel and installs it at "${KERNEL_OUTPUT}". Runs on the
# host, as yourself: nothing here needs root.
#
# The tree is CachyOS's fork, taken for its patches rather than its config.
# Theirs is a desktop distro config with thousands of modules; this guest has
# no /lib/modules at all. What must hold is kernel/nestri.fragment, merged onto
# whatever .config the tree has and then verified, so a version bump that
# quietly drops an option fails here instead of in a booted box.
set -euo pipefail
: "${KERNEL_GIT:?}"
: "${KERNEL_REF:?}"
: "${KERNEL_SRC:?}"
: "${KERNEL_OUTPUT:?}"
JOBS="${JOBS:-$(nproc)}"
KERNEL_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../kernel" && pwd)"
FRAGMENT="${KERNEL_DIR}/nestri.fragment"
SEED="${KERNEL_DIR}/base.config"
# Resolved now, because everything below runs from inside the tree.
mkdir -p "$(dirname "${KERNEL_OUTPUT}")"
KERNEL_OUTPUT="$(cd "$(dirname "${KERNEL_OUTPUT}")" && pwd)/$(basename "${KERNEL_OUTPUT}")"
# ── Source ──────────────────────────────────────────────
if [[ ! -f "${KERNEL_SRC}/Makefile" ]]; then
echo "kernel: cloning ${KERNEL_REF} into ${KERNEL_SRC}"
mkdir -p "$(dirname "${KERNEL_SRC}")"
git clone --depth=1 --branch "${KERNEL_REF}" "${KERNEL_GIT}" "${KERNEL_SRC}"
else
have="$(git -C "${KERNEL_SRC}" describe --tags --exact-match 2>/dev/null || echo unknown)"
if [[ "${have}" != "${KERNEL_REF}" ]]; then
# Not fatal: a bisect or a local patch is a legitimate reason to be off
# the pinned ref, and silently checking it out would throw that away.
echo "kernel: tree is at '${have}', KERNEL_REF pins '${KERNEL_REF}'; building what is there" >&2
fi
fi
cd "${KERNEL_SRC}"
# ── Infinity scheduler (experimental) ───────────────────
# Applied once per tree, and recorded, because `patch -N` on an already patched
# tree does not skip cleanly: it rejects every hunk. The whole series goes in or
# none of it does -- upstream is explicit that a partial series misbehaves -- so
# every patch is dry-run against the stacked result before any is applied.
#
# In this guest only the fair and rt halves do anything. virtio-gpu does not
# use the DRM scheduler, so the gpu patch is compiled out with the rest of
# drivers/gpu/drm/scheduler; it is applied anyway to keep the series whole.
if [[ -n "${KERNEL_INFINITY:-}" ]]; then
: "${INFINITY_GIT:?}" "${INFINITY_REV:?}" "${INFINITY_SERIES:?}" "${INFINITY_WORK:?}"
stamp=".nestri-infinity-rev"
if [[ "$(git -C "${INFINITY_WORK}" rev-parse HEAD 2>/dev/null)" != "${INFINITY_REV}" ]]; then
echo "kernel: fetching infinity-sched ${INFINITY_REV}"
rm -rf "${INFINITY_WORK}"
git init -q "${INFINITY_WORK}"
git -C "${INFINITY_WORK}" fetch -q --depth=1 "${INFINITY_GIT}" "${INFINITY_REV}"
git -C "${INFINITY_WORK}" checkout -q FETCH_HEAD
fi
series_dir="${INFINITY_WORK}/${INFINITY_SERIES}"
[[ -f "${series_dir}/series" ]] || {
echo "kernel: infinity-sched has no series at ${INFINITY_SERIES} for ${KERNEL_REF}" >&2
exit 1
}
have="$(cat "${stamp}" 2>/dev/null || true)"
if [[ "${have}" == "${INFINITY_REV}" ]]; then
echo "kernel: infinity series already applied"
elif [[ -n "${have}" ]]; then
echo "kernel: tree carries infinity ${have}, INFINITY_REV pins ${INFINITY_REV}" >&2
echo "kernel: start the tree over with \`make kernel-clean\`" >&2
exit 1
else
if [[ -n "$(git status --porcelain --untracked-files=no)" ]]; then
echo "kernel: ${KERNEL_SRC} has local changes; not applying the series over them" >&2
exit 1
fi
mapfile -t patches < <(grep -v '^[[:space:]]*\(#\|$\)' "${series_dir}/series")
# git apply --check takes the whole list and checks each patch against
# the result of the ones before it, which a per-file `patch --dry-run`
# cannot do.
git apply --check "${patches[@]/#/${series_dir}/}"
for p in "${patches[@]}"; do
echo "kernel: applying ${p}"
# -F 0: zero fuzz. Offsets are fine; a hunk that only fits
# approximately is a scheduler change landing somewhere it was not
# written for.
patch -p1 -N -F 0 --quiet < "${series_dir}/${p}"
done
echo "${INFINITY_REV}" > "${stamp}"
fi
fi
# ── Config ──────────────────────────────────────────────
# A fresh tree has no .config. The seed is a known-good minimal config that
# olddefconfig migrates to whatever version the tree is at; it only saves a
# fresh tree from `make defconfig`, whose driver set is enormous next to what a
# microVM needs. An existing .config is always preferred.
if [[ ! -f .config ]]; then
echo "kernel: seeding .config from kernel/base.config"
cp "${SEED}" .config
fi
echo "kernel: merging kernel/nestri.fragment"
# -m merges without running a config target, so olddefconfig resolves
# dependencies once, in one place.
./scripts/kconfig/merge_config.sh -m .config "${FRAGMENT}" >/dev/null
make olddefconfig >/dev/null
# ── Verify the fragment actually took ───────────────────
# merge_config.sh warns about overridden symbols but exits 0, and olddefconfig
# will happily drop an option whose dependencies are unmet. Neither is loud
# enough for a setting whose failure mode is silent audio, so check the result
# rather than the intent. Both halves count: an option that must be on, and one
# that must be off.
missing=()
total=0
while read -r want; do
total=$((total + 1))
case "${want}" in
CONFIG_*) grep -qx "${want}" .config || missing+=("${want%%=*}") ;;
"# "*) grep -qx "${want}" .config || missing+=("${want:2} (must be off)") ;;
esac
done < <(grep -E '^(CONFIG_[A-Z0-9_]+=|# CONFIG_[A-Z0-9_]+ is not set)' "${FRAGMENT}" \
| sed -E 's/^(CONFIG_[A-Z0-9_]+=[^[:space:]#]+)[[:space:]]*#.*/\1/')
if (( ${#missing[@]} )); then
echo "kernel: these fragment entries did not survive olddefconfig:" >&2
printf '%s\n' "${missing[@]}" >&2
exit 1
fi
echo "kernel: all ${total} fragment entries hold"
# ── Build ───────────────────────────────────────────────
# -march goes in through KCFLAGS because mainline has no Kconfig for
# microarchitecture levels. It is safe for the kernel even at x86-64-v3:
# arch/x86/Makefile passes -mno-sse -mno-mmx -mno-sse2 -mno-avx and friends,
# and gcc applies those as a mask over -march regardless of flag order, so the
# kernel gets v3's integer ISA (BMI2, LZCNT, MOVBE) and its scheduling model
# and never touches a vector register.
make_args=()
if [[ -n "${KERNEL_MARCH:-}" ]]; then
make_args+=("KCFLAGS=-march=${KERNEL_MARCH}")
echo "kernel: building with -march=${KERNEL_MARCH}"
fi
# vmlinux, not bzImage: the guest is booted by an ELF loader with no
# bootloader in the path, and a bzImage is a self-decompressing image behind a
# real-mode setup header, not an ELF.
make -j"${JOBS}" "${make_args[@]}" vmlinux
cp vmlinux "${KERNEL_OUTPUT}"
echo "kernel: installed ${KERNEL_OUTPUT} ($(numfmt --to=iec "$(stat -c %s "${KERNEL_OUTPUT}")"))"
build/scripts/proton-build.sh
+137 -83
View File
@@ -1,96 +1,150 @@
#!/usr/bin/env bash
# Builds proton-cachyos from the tree proton-fetch.sh laid down. Container-only.
# Builds proton-ge wow64-only and leaves the finished tree in "${PROTON_WORK}/obj/dist".
# Runs on the host, not in a container.
#
# The one thing that matters here is --enable-wow64: it builds wine so that
# 32-bit Windows code runs inside a 64-bit unix process, thunking down to the
# 64-bit host libraries. Without it, Proton needs a complete 32-bit host stack —
# lib32 glibc, a second Mesa built for i686, and a second nescapture layer,
# because a 32-bit game would load the 32-bit Vulkan loader and our 64-bit
# capture layer would be invisible to it. With it, none of that exists.
# It has to run on the host because proton-ge's build is itself container-driven:
# `make` runs outside, and every step runs in the Steam Runtime SDK image, where
# the toolchains live, through the engine it is configured with. There is no
# mode without a container, and a container engine inside `podman build` is
# nested containers, which is a lot of fragile setup for no gain. So the only
# thing this script needs from the host is git, make and the engine. The
# Makefile packages the result afterwards.
#
# The cost is that the distro package cannot be used: proton-cachyos-native is
# packaged without the flag, which is exactly why it depends on lib32-*.
# The one thing we change is the arch list: it becomes wow64-only, and that
# change is the reason this is our own build and not a download. wow64 runs
# 32-bit Windows code inside a 64-bit unix process. Without it Proton needs a
# complete 32-bit host stack: lib32 glibc, a second Mesa built for i686, and a
# second nescapture layer, because a 32-bit game would load the 32-bit Vulkan
# loader and our 64-bit capture layer would be invisible to it. The released
# builds carry an i386 unix side, which is exactly why they need lib32-*.
#
# Everything else is proton-ge's own recipe: the same SDK image, the same flags
# and the same patch set. The one addition is patches/proton-ge/: fixes for the
# places its makefile assumes a 32-bit unix side that wow64 does not have, and
# for things a tag pinned that have since moved out from under it.
set -euo pipefail
: "${GECKO_VER:?}"
: "${MONO_VER:?}"
: "${PROTON_GIT:?}"
: "${PROTON_TAG:?}"
: "${PROTON_WORK:?}"
: "${BUILD_NAME:?}"
ENGINE="${CONTAINER_ENGINE:-podman}"
JOBS="${JOBS:-$(nproc)}"
BUILD_NAME="proton-cachyos"
SRC_DIR="/build/proton-cachyos"
BUILD_DIR="/build/build"
OUT_DIR="/artifacts/proton/usr/share/steam/compatibilitytools.d/${BUILD_NAME}"
[[ -d "${SRC_DIR}" ]] || { echo "no source tree — proton-fetch.sh did not run"; exit 1; }
mkdir -p "${PROTON_WORK}"
PROTON_WORK="$(cd "${PROTON_WORK}" && pwd)"
SRC="${PROTON_WORK}/src"
OBJ="${PROTON_WORK}/obj"
STAMP_TAG="${PROTON_WORK}/.tag"
STAMP_PATCHED="${PROTON_WORK}/.patched"
PATCH_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../patches/proton-ge" && pwd)"
# ── Toolchain wrappers ──────────────────────────────────
# Proton's build calls the compiler by GNU triplet. Arch's gcc does not install
# under those names, so stand in for them. The i686 set is generated too: with
# wow64 nothing should reach for it, and if something does, failing on a missing
# 32-bit header beats silently building a 32-bit unix library we then have to
# ship libraries for.
WRAP=/build/wrappers
rm -rf "$WRAP" && mkdir -p "$WRAP"
_wrappers() {
local arch="$1" gccflag="$2" ldflag="$3" asflag="$4" stripfmt="$5"
local l t
for l in ar ranlib nm; do
ln -sf "/usr/bin/gcc-${l}" "${WRAP}/${arch}-pc-linux-gnu-${l}"
# Two builds in one tree do not fail cleanly. They race on the same objects and
# leave half-written files that a later build trusts. A failed make also keeps
# running its in-flight jobs for a while after it reports the error, so the
# first build is often still running when the second one starts.
exec 9>"${PROTON_WORK}/.lock"
flock -n 9 || { echo "proton: another build is using ${PROTON_WORK}" >&2; exit 1; }
# ccache and cargo's downloads are kept outside src/ and obj/, so a new tag or
# FORCE_REBUILD throws away the build and keeps the parts that are correct to
# reuse. proton-ge's makefile mounts both into the container from these
# variables.
export CCACHE_DIR="${PROTON_WORK}/ccache"
export CARGO_HOME="${PROTON_WORK}/cargo"
mkdir -p "${CCACHE_DIR}" "${CARGO_HOME}"
if [[ -n "${FORCE_REBUILD:-}" || "$(cat "${STAMP_TAG}" 2>/dev/null)" != "${PROTON_TAG}" ]]; then
echo "proton: fresh tree for ${PROTON_TAG}"
rm -rf "${SRC}" "${OBJ}" "${STAMP_TAG}" "${STAMP_PATCHED}"
fi
# ── Fetch ───────────────────────────────────────────────
if [[ ! -e "${STAMP_TAG}" ]]; then
rm -rf "${SRC}"
git clone --branch "${PROTON_TAG}" --depth=1 "${PROTON_GIT}" "${SRC}"
# No --depth here: submodules are pinned to commits that are often not a
# branch tip. --filter=tree:0 keeps the download down instead.
git -C "${SRC}" submodule update --init --filter=tree:0 --recursive
echo "${PROTON_TAG}" > "${STAMP_TAG}"
fi
# The SDK image is pinned by proton-ge's own makefile, per tag. Asking it keeps
# the patch step below and the build on the same image.
SDK_IMAGE="$(make --silent --no-print-directory -f "${SRC}/Makefile.in" \
SRCDIR="${SRC}" get-steamrt-image)"
# ── Patch ───────────────────────────────────────────────
# The patch script edits the tree in place and is not idempotent: it resets
# some submodules first and not others. So a tree is patched once, and one that
# was interrupted halfway is reset to the commits the tag pins before trying
# again.
#
# It is run in the SDK image rather than on the host, so it does not depend on
# the host's python, patch or wget.
#
# The script carries on past a patch that does not apply and exits 0 anyway.
# The upstream instructions are to grep its output for failures, so that is
# what happens here. The alternative is an image that looks fine and is missing
# a fix.
#
# A build tree does not survive its source being re-patched. Changing
# Makefile.in re-syncs every component's source copy, but a component's
# configure step depends on that sync order-only, so it does not rerun, and its
# old build directory is left pointing at generated autotools files the sync
# just removed. So patching starts obj/ over too. ccache keeps that cheap.
if [[ ! -e "${STAMP_PATCHED}" ]]; then
rm -rf "${OBJ}"
git -C "${SRC}" reset -q --hard
git -C "${SRC}" submodule foreach -q --recursive 'git reset -q --hard && git clean -qfdx'
"${ENGINE}" run --rm -v "${SRC}:${SRC}" -w "${SRC}" "${SDK_IMAGE}" \
./patches/protonprep-valve-staging.sh 2>&1 | tee "${PROTON_WORK}/patch.log"
if grep -Ei 'hunk .* failed|saving rejects|can.t find file|malformed patch|skipping patch|^error' \
"${PROTON_WORK}/patch.log"; then
echo "proton: patches did not apply cleanly, see ${PROTON_WORK}/patch.log" >&2
exit 1
fi
# Ours go on top. They are paths from the root of the tree, submodules
# included. `git apply` fails outright on a patch that no longer applies,
# which is what a tag bump should do: each one says why it exists, so the
# question is only whether upstream fixed it.
for p in "${PATCH_DIR}"/*.patch; do
[[ -e "$p" ]] || continue
echo "proton: applying $(basename "$p")"
git -C "${SRC}" apply "$p"
done
for t in gcc g++; do
printf '#!/usr/bin/bash\n/usr/bin/%s %s "$@"\n' "$t" "$gccflag" \
> "${WRAP}/${arch}-pc-linux-gnu-${t}"
chmod 755 "${WRAP}/${arch}-pc-linux-gnu-${t}"
done
printf '#!/usr/bin/bash\n/usr/bin/ld %s "$@"\n' "$ldflag" > "${WRAP}/${arch}-pc-linux-gnu-ld"
printf '#!/usr/bin/bash\n/usr/bin/as %s "$@"\n' "$asflag" > "${WRAP}/${arch}-pc-linux-gnu-as"
printf '#!/usr/bin/bash\n/usr/bin/strip -F %s "$@"\n' "$stripfmt" > "${WRAP}/${arch}-pc-linux-gnu-strip"
chmod 755 "${WRAP}/${arch}-pc-linux-gnu-"{ld,as,strip}
}
_wrappers x86_64 "-m64" "-melf_x86_64" "--64" "elf64-x86-64"
_wrappers i686 "-m32" "-melf_i386" "--32" "elf32-i386"
export PATH="${WRAP}:${PATH}"
touch "${STAMP_PATCHED}"
fi
# ── Configure ───────────────────────────────────────────
# configure.sh refuses an in-tree build, and it test-runs the SDK image to work
# out how the engine maps file ownership, so it is also where a broken engine
# setup shows up first.
mkdir -p "${OBJ}"
if [[ ! -e "${OBJ}/Makefile" ]]; then
(cd "${OBJ}" && "${SRC}/configure.sh" \
--build-name="${BUILD_NAME}" \
--container-engine="${ENGINE}")
fi
# ── Build ───────────────────────────────────────────────
# -march=nocona matches the distro packaging: Proton has to run on whatever CPU
# the guest is given, and the VMM does not promise a feature level.
export CFLAGS="-O3 -march=nocona -mtune=core-avx2"
export CXXFLAGS="${CFLAGS}"
export RUSTFLAGS="-C opt-level=3 -C target-cpu=nocona"
export LDFLAGS="-Wl,-O1,--sort-common,--as-needed"
export RUSTUP_TOOLCHAIN=stable
# ARCHS drops i386-unix, which leaves wine configured for x86_64 unix with an
# i386 PE side. That is wow64. Every component rule is gated on ARCHS, so the
# 32-bit unix builds of everything else go with it. ENABLE_WOW64 makes the
# proton script ask wine for a wow64 prefix. proton-ge ships it as a switch
# but never turns it on.
#
# A command-line variable reaches the container build too: the outer make
# hands its overrides to the inner one.
#
# SOURCE_DATE_EPOCH is the tag's commit time rather than now, so two builds of
# one tag stamp the same dates into their output.
make -C "${OBJ}" \
J="${JOBS}" \
ARCHS="i386-windows x86_64-windows x86_64-unix" \
ENABLE_WOW64=1 \
SOURCE_DATE_EPOCH="$(git -C "${SRC}" log -1 --format=%ct)" \
dist
mkdir -p "${BUILD_DIR}"
cd "${BUILD_DIR}"
ROOTLESS_CONTAINER="" \
"${SRC_DIR}/configure.sh" \
--container-engine="none" \
--proton-sdk-image="" \
--build-name="${BUILD_NAME}" \
--without-extras=all \
--without-vklayers=all \
--without-steamrt-depends \
--without-tts \
--without-nvidia-libs \
--enable-wow64
# The top-level make is serial by design; SUBJOBS is what it hands to each
# component's build.
SUBJOBS="${JOBS}" make -j1 dist
# ── Install ─────────────────────────────────────────────
mkdir -p "${OUT_DIR}"
cp -a "${BUILD_DIR}/dist/." "${OUT_DIR}/"
# Debug symbols in the bundled PE runtimes are dead weight in a guest image.
cd "${OUT_DIR}/files"
find "share/wine/gecko/wine-gecko-${GECKO_VER}-x86" -name '*.dll' -o -name '*.exe' 2>/dev/null \
| xargs -r i686-w64-mingw32-strip --strip-debug 2>/dev/null || true
find "share/wine/gecko/wine-gecko-${GECKO_VER}-x86_64" -name '*.dll' -o -name '*.exe' 2>/dev/null \
| xargs -r x86_64-w64-mingw32-strip --strip-debug 2>/dev/null || true
find "share/wine/mono/wine-mono-${MONO_VER}" -name '*.dll' -o -name '*.exe' 2>/dev/null \
| xargs -r i686-w64-mingw32-strip --strip-debug 2>/dev/null || true
rm -rf "${BUILD_DIR}"
echo "proton: installed to ${OUT_DIR}"
echo "proton: built ${OBJ}/dist"
build/scripts/proton-fetch.sh
-57
View File
@@ -1,57 +0,0 @@
#!/usr/bin/env bash
# Fetches proton-cachyos' source and its bundled runtimes. Container-only.
#
# Deliberately its own script, and its own layer: the submodule checkout runs
# well past ten minutes, and it must not be redone every time a build flag or a
# missing dependency changes. Keep everything that can fail *fast* in
# proton-build.sh instead.
set -euo pipefail
: "${PROTON_GIT:?}"
: "${PROTON_TAG:?}"
: "${GECKO_VER:?}"
: "${MONO_VER:?}"
: "${XALIA_VER:?}"
SRC_DIR="/build/proton-cachyos"
git clone --branch "${PROTON_TAG}" --depth=1 "${PROTON_GIT}" "${SRC_DIR}"
cd "${SRC_DIR}"
# Relative submodule paths resolve against origin, so it has to be the real URL
# even though we cloned by tag.
git remote set-url origin "${PROTON_GIT}"
# No --depth here: submodules are pinned to commits that are often not a branch
# tip. --filter=tree:0 keeps the download down instead.
git submodule update --init --filter=tree:0 --recursive
# Still needed with wow64: these are PE-side, and a 32-bit Windows program wants
# the 32-bit gecko and mono regardless of how wine is built.
mkdir -p contrib
for url in \
"https://dl.winehq.org/wine/wine-gecko/${GECKO_VER}/wine-gecko-${GECKO_VER}-x86.tar.xz" \
"https://dl.winehq.org/wine/wine-gecko/${GECKO_VER}/wine-gecko-${GECKO_VER}-x86_64.tar.xz" \
"https://github.com/madewokherd/wine-mono/releases/download/wine-mono-${MONO_VER}/wine-mono-${MONO_VER}-x86.tar.xz" \
"https://github.com/madewokherd/xalia/releases/download/xalia-${XALIA_VER}/xalia-${XALIA_VER}-net48-mono.zip" \
; do
curl -fL --retry 3 -o "contrib/$(basename "$url")" "$url"
done
# Proton's cargo rule runs `cargo build --locked --offline`, so every crate has
# to be in CARGO_HOME before the build starts — including the git dependencies,
# which is what the "you are in the offline mode" failure is really saying. The
# error names a URL that is perfectly reachable; the build simply refuses to go
# out and get it.
#
# gst-plugins-rs is the only cargo component in the tree. Both targets are
# fetched: wow64 should mean nothing builds the i386 unix side, but a fetch is
# metadata only and costs almost nothing next to being wrong about that.
#
# CARGO_HOME is left at its default so it lands in this layer and the build
# layer inherits it.
export CARGO_NET_GIT_FETCH_WITH_CLI=true
export RUSTUP_TOOLCHAIN=stable
cd "${SRC_DIR}/gst-plugins-rs"
cargo fetch --locked --target x86_64-unknown-linux-gnu
cargo fetch --locked --target i686-unknown-linux-gnu
echo "proton: source at ${SRC_DIR}"
crates/nesprotocol/src/delay.rs
+327
View File
@@ -0,0 +1,327 @@
//! How much a delay varies, measured the same way at both ends.
//!
//! # What this measures
//!
//! Every frame carries the timestamp the sender stamped on it at capture. The
//! two clocks are unrelated, so the difference between that and the moment the
//! frame arrives here is meaningless on its own -- it contains an unknown,
//! roughly constant offset. Its *variation* is not meaningless, and variation is
//! the only thing a playout buffer exists to absorb: a path that delivers every
//! frame exactly 300 ms late needs no buffer at all, while one that alternates
//! between 10 ms and 60 ms needs 50 ms whatever its average.
//!
//! So this tracks the smallest difference seen recently -- the best the path has
//! managed, which stands in for the unknown offset -- and reports how far above
//! it each frame lands.
//!
//! # What it deliberately does not do
//!
//! It does not distinguish the sender's contribution from the network's. The
//! sender reports its own pipeline delay separately, and the two are compared
//! rather than subtracted: a frame that was late because the encoder stalled
//! should not raise a buffer, because buffering is latency spent hiding a fault
//! that should be fixed instead.
//!
//! # Why this is shared
//!
//! Both ends measure a delay and the two are compared: the hub reports how much
//! its own pipeline varied before a frame left, the client reports how much the
//! total varied by the time it arrived. A comparison between two different
//! measures would be meaningless, so there is one measure, defined once.
//!
//! It also has to be *variation* on both sides rather than absolute delay,
//! because neither side can know the absolute. The hub's `timestamp_ms` counts
//! from the encoder's own start, not from any epoch, so even on one machine the
//! difference to wall clock contains an unknown constant -- and across two
//! machines there is no shared clock at all.
//!
//! Nothing here adapts anything yet. It measures, so that the decision about
//! what to adapt is made against a distribution rather than an intuition.
use std::collections::VecDeque;
use std::time::{Duration, Instant};
/// How long a best-case observation stays authoritative.
///
/// The minimum has to expire. Paths change -- a relay is dropped for a direct
/// connection, a phone moves between cells -- and a minimum from the old path
/// makes every frame on the new one look permanently late, which would pin a
/// buffer at a size nothing needs. Long enough that an ordinary quiet spell does
/// not reset the baseline, short enough that a genuine change is noticed.
const BASELINE_WINDOW: Duration = Duration::from_secs(30);
/// Ceiling on retained observations, so a stalled consumer cannot grow this.
const MAX_SAMPLES: usize = 4096;
/// One second's worth of lateness, summarised.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct DelaySummary {
/// Frames measured.
pub frames: u32,
/// Median lateness above the best the path has managed, in milliseconds.
pub p50_ms: u16,
/// The 95th percentile of the same.
pub p95_ms: u16,
/// The worst single frame.
pub max_ms: u16,
}
/// Tracks arrival lateness for one media stream.
#[derive(Debug)]
pub struct DelayTracker {
/// Best-case delay observations, each with when it was taken, oldest first.
///
/// A deque rather than a single value because a minimum that can only fall
/// never recovers from a path that improved, and one that is simply reset on
/// a timer throws away a good baseline for no reason. Holding the recent
/// candidates lets the oldest expire while a better one is still standing.
baseline: VecDeque<(Instant, i32)>,
/// Lateness of each frame this second, in milliseconds.
samples: Vec<u16>,
/// The first delay seen, as an anchor for every later one.
///
/// Delays are compared *relative to this*, in wrapping arithmetic. Both
/// clocks are unrelated and the sender's stamp is a `u32` of milliseconds
/// that wraps every 49 days, so an absolute subtraction is a number with no
/// meaning and a wrap in the middle of a session makes it jump by 2^32 --
/// which, taken as lateness, reads as every frame being weeks late until the
/// baseline expires. Anchoring and then interpreting the difference as
/// signed makes the wrap a non-event, because the quantity that matters is
/// only ever a few hundred milliseconds wide.
anchor: Option<u32>,
}
impl Default for DelayTracker {
fn default() -> Self {
Self::new()
}
}
impl DelayTracker {
pub fn new() -> Self {
Self {
baseline: VecDeque::new(),
samples: Vec::new(),
anchor: None,
}
}
/// Record one frame's arrival.
///
/// `stamp_ms` is the timestamp the frame carries; `observed_ms` is the local
/// clock at the point being measured -- arrival, for a receiver; the moment
/// the frame is handed to the transport, for a sender. See [`anchor`](Self::anchor) for why the difference is taken
/// in wrapping arithmetic rather than widened.
pub fn observe(&mut self, stamp_ms: u32, observed_ms: u64, now: Instant) {
let raw = (observed_ms as u32).wrapping_sub(stamp_ms);
let anchor = *self.anchor.get_or_insert(raw);
let delay = raw.wrapping_sub(anchor) as i32;
self.expire(now);
// Anything at or below the running minimum becomes the new baseline, and
// supersedes the candidates it beats -- they can only be worse, so
// keeping them would let a stale, higher value resurface on expiry.
while self.baseline.back().is_some_and(|(_, d)| *d >= delay) {
self.baseline.pop_back();
}
self.baseline.push_back((now, delay));
let best = self.baseline.front().map(|(_, d)| *d).unwrap_or(delay);
let lateness = i64::from(delay)
.saturating_sub(i64::from(best))
.clamp(0, i64::from(u16::MAX)) as u16;
if self.samples.len() < MAX_SAMPLES {
self.samples.push(lateness);
}
}
/// Summarise and clear the frames seen since the last call.
///
/// Returns `None` for a second in which nothing arrived. That is not the
/// same as a second with no lateness, and reporting zero would say the path
/// is behaving perfectly at the moment it has stopped delivering anything.
pub fn take(&mut self) -> Option<DelaySummary> {
if self.samples.is_empty() {
return None;
}
self.samples.sort_unstable();
let at = |q: f64| self.samples[((self.samples.len() - 1) as f64 * q) as usize];
let summary = DelaySummary {
frames: self.samples.len() as u32,
p50_ms: at(0.5),
p95_ms: at(0.95),
max_ms: self.samples[self.samples.len() - 1],
};
self.samples.clear();
Some(summary)
}
/// Forget the baseline entirely.
///
/// For a path change, where the old best case says nothing about the new
/// path and keeping it would make every frame look late until it expired.
pub fn reset_baseline(&mut self) {
self.baseline.clear();
}
fn expire(&mut self, now: Instant) {
while self
.baseline
.front()
.is_some_and(|(seen, _)| now.duration_since(*seen) > BASELINE_WINDOW)
{
self.baseline.pop_front();
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A frame sent at `ts_ms` arriving `transit` milliseconds later, where the
/// receiver's clock is offset from the sender's by an arbitrary amount.
const OFFSET: u64 = 1_700_000_000_000;
fn at(j: &mut DelayTracker, ts_ms: u32, transit: u64, now: Instant) {
j.observe(ts_ms, OFFSET + u64::from(ts_ms) + transit, now);
}
#[test]
fn a_constant_delay_is_not_jitter() {
// The point of the whole measure. A path that delivers every frame
// exactly 300 ms late needs no buffer at all; only variation does.
let mut j = DelayTracker::new();
let now = Instant::now();
for i in 0..120u32 {
at(
&mut j,
i * 16,
300,
now + Duration::from_millis(u64::from(i) * 16),
);
}
let s = j.take().expect("frames arrived");
assert_eq!(s.max_ms, 0, "a constant offset was read as lateness");
assert_eq!(s.frames, 120);
}
#[test]
fn lateness_is_measured_against_the_best_the_path_managed() {
let mut j = DelayTracker::new();
let now = Instant::now();
at(&mut j, 0, 20, now);
at(&mut j, 16, 70, now + Duration::from_millis(16));
at(&mut j, 32, 20, now + Duration::from_millis(32));
let s = j.take().expect("frames arrived");
assert_eq!(
s.max_ms, 50,
"the 70 ms frame is 50 ms above the 20 ms best"
);
}
#[test]
fn a_silent_second_reports_nothing_rather_than_no_jitter() {
// Zero would say the path is behaving perfectly at the moment it has
// stopped delivering anything at all.
let mut j = DelayTracker::new();
assert_eq!(j.take(), None);
}
#[test]
fn the_baseline_expires_so_an_improved_path_is_noticed() {
// A minimum that can only fall never recovers: one lucky early frame
// would make every later frame look late for the rest of the session.
let mut j = DelayTracker::new();
let now = Instant::now();
at(&mut j, 0, 10, now);
j.take();
// Much later, the path settles at a steady 200 ms.
let later = now + BASELINE_WINDOW + Duration::from_secs(1);
for i in 0..10u32 {
at(
&mut j,
1000 + i * 16,
200,
later + Duration::from_millis(u64::from(i) * 16),
);
}
let s = j.take().expect("frames arrived");
assert_eq!(
s.max_ms, 0,
"a stale best case from a previous path made a steady path look late",
);
}
#[test]
fn a_better_observation_supersedes_worse_ones_still_in_the_window() {
// Otherwise a higher candidate resurfaces when the better one expires,
// and the baseline walks upwards for no reason the path can account for.
let mut j = DelayTracker::new();
let now = Instant::now();
at(&mut j, 0, 90, now);
at(&mut j, 16, 10, now + Duration::from_millis(16));
// The 90 ms candidate is gone, so this sits 40 ms above the 10 ms best.
at(&mut j, 32, 50, now + Duration::from_millis(32));
let s = j.take().expect("frames arrived");
assert_eq!(s.max_ms, 40);
}
#[test]
fn a_reset_forgets_the_old_path_entirely() {
let mut j = DelayTracker::new();
let now = Instant::now();
at(&mut j, 0, 10, now);
j.take();
j.reset_baseline();
at(&mut j, 16, 400, now + Duration::from_millis(16));
let s = j.take().expect("frames arrived");
assert_eq!(s.max_ms, 0, "the new path was judged against the old one");
}
#[test]
fn the_tail_is_kept_apart_from_the_middle() {
// The number that sizes a buffer is the tail. A run of prompt frames
// with one bad one must not average into "slightly late".
let mut j = DelayTracker::new();
let now = Instant::now();
for i in 0..99u32 {
at(
&mut j,
i * 16,
10,
now + Duration::from_millis(u64::from(i) * 16),
);
}
at(&mut j, 99 * 16, 260, now + Duration::from_millis(99 * 16));
let s = j.take().expect("frames arrived");
assert_eq!(s.p50_ms, 0);
assert_eq!(s.max_ms, 250);
}
#[test]
fn taking_clears_so_each_answer_describes_one_second() {
let mut j = DelayTracker::new();
let now = Instant::now();
at(&mut j, 0, 10, now);
assert!(j.take().is_some());
assert_eq!(
j.take(),
None,
"a second reported the previous second again"
);
}
#[test]
fn a_wrapped_sender_timestamp_does_not_poison_the_baseline() {
// ts_ms wraps every 49 days. A negative delay taken as the best case
// would make every subsequent frame appear weeks late.
let mut j = DelayTracker::new();
let now = Instant::now();
j.observe(u32::MAX - 10, OFFSET, now);
j.observe(5, OFFSET + 16, now + Duration::from_millis(16));
let s = j.take().expect("frames arrived");
assert!(s.max_ms < u16::MAX, "a wrap produced a nonsense lateness");
}
}
crates/nesprotocol/src/lib.rs
+719 -4
View File
@@ -3,13 +3,64 @@
// can drift from each other silently.
pub mod datagram;
pub mod delay;
pub mod input;
#[cfg(feature = "lifecycle")]
pub mod lifecycle;
pub mod reliable;
pub mod stats;
pub const ALPN: &[u8] = b"/nestri/stream/1";
/// One connection per kind of traffic, not one connection for everything.
///
/// A QUIC connection is the unit that congestion control, pacing and the
/// datagram send buffer all operate on, so everything sharing one shares a
/// queue. Measured over a 1000-mile link: a video backlog delayed audio and
/// input with it, because a backlog is a property of the connection and video
/// is the only flow large enough to build one. Audio behind a deep video queue
/// was twenty-five times worse for timing than audio behind a shallow one --
/// the same audio, on the same path, ruined by what it was queued behind.
///
/// Splitting them gives each its own congestion controller and its own send
/// buffer, so video can only ever delay video. They compete at a shared
/// bottleneck rather than cooperating, which is the point: audio and input are
/// small and need a share, not a place in line behind a keyframe.
pub const ALPN_VIDEO: &[u8] = b"/nestri/video/1";
pub const ALPN_AUDIO: &[u8] = b"/nestri/audio/1";
pub const ALPN_INPUT: &[u8] = b"/nestri/input/1";
pub const ALPN_CONTROL: &[u8] = b"/nestri/control/1";
/// Every ALPN a hub accepts, for the endpoint builder.
pub const ALPNS: [&[u8]; 4] = [ALPN_VIDEO, ALPN_AUDIO, ALPN_INPUT, ALPN_CONTROL];
/// Which connection an accepted one is, by its ALPN.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Carrier {
Video,
Audio,
Input,
Control,
}
impl Carrier {
pub fn from_alpn(alpn: &[u8]) -> Option<Self> {
match alpn {
a if a == ALPN_VIDEO => Some(Self::Video),
a if a == ALPN_AUDIO => Some(Self::Audio),
a if a == ALPN_INPUT => Some(Self::Input),
a if a == ALPN_CONTROL => Some(Self::Control),
_ => None,
}
}
pub fn label(&self) -> &'static str {
match self {
Self::Video => "video",
Self::Audio => "audio",
Self::Input => "input",
Self::Control => "control",
}
}
}
pub const IPC_MAGIC: [u8; 4] = [b'N', b'S', b'T', b'R'];
@@ -38,8 +89,21 @@ pub const FRAME_HDR_LEN: usize = 7;
/// For single-purpose streams (video, audio) the type is redundant but included
/// for uniformity.
pub const MSG_DATA: u8 = 0; // generic data frame (video / audio)
pub const MSG_IDR_REQUEST: u8 = 0x10; // request a keyframe (desktop → hub → hudless)
pub const MSG_ENCODE_SETTINGS: u8 = 0x12; // change encoder settings (desktop → hub → hudless)
pub const MSG_IDR_REQUEST: u8 = 0x10; // request a keyframe (desktop → hub → nescapture)
pub const MSG_ENCODE_SETTINGS: u8 = 0x12; // change encoder settings (desktop → hub → nescapture)
pub const MSG_CLIENT_CAPS: u8 = 0x15; // what the client can decode (desktop → hub → nescapture)
pub const MSG_SURFACE_COLOR: u8 = 0x16; // what the compositor was told a surface is (nescope → nescapture)
/// What the receiver actually got, once a second (desktop → hub).
///
/// The hub cannot see this. Its own view of the path -- RTT, congestion window,
/// whether a datagram send returned an error -- was measured saying the path was
/// healthy while the client was receiving almost nothing, and one reason is
/// structural: `send_datagram` evicts the oldest queued datagrams and returns
/// `Ok`, so the send side has no backpressure signal at all. Only the far end
/// knows what arrived.
pub const MSG_RECEIVER_REPORT: u8 = 0x13;
/// Who decides the bitrate, and the ceiling to decide within (desktop → hub).
pub const MSG_CONTROL_MODE: u8 = 0x14;
pub const MSG_INPUT_BATCH: u8 = 0xFE; // batched input events (desktop → hub)
/// Build a frame body: `[u8 type] [u16 LE seq] [payload]`.
@@ -75,7 +139,7 @@ pub fn decode_frame(frame: &[u8]) -> Option<(u8, u16, &[u8])> {
// ── Stream types ────────────────────────────────────────────────
//
// These name a media kind, not a transport. `STREAM_VIDEO` and `STREAM_AUDIO`
// still tag IPC frames on the hudless→hub unix sockets, but over QUIC their
// still tag IPC frames on the nescapture→hub unix sockets, but over QUIC their
// media now travels as datagrams; only cursor and stats still open a uni
// stream and send this as its first byte.
@@ -90,6 +154,13 @@ pub const STREAM_KEYFRAME: u8 = 5;
// ── Bidi stream types (desktop ↔ hub over QUIC bidi) ───────────
pub const BIDI_INPUT: u8 = 2; // desktop → hub → nescope (input events)
/// Everything the client says that is not an input event: keyframe requests,
/// receiver reports, encode settings, control mode.
///
/// Its own stream on its own connection. Input is small and latency-critical
/// and must not wait behind a receiver report, and neither must wait behind
/// video, which is why these live apart from the media connections entirely.
pub const BIDI_CONTROL: u8 = 6;
// Codec IDs
pub const CODEC_H264: u8 = 0;
@@ -101,6 +172,13 @@ pub const CODEC_KEEP: u8 = 0xFF; // "keep current" sentinel for dynamic encoder
// Rate control modes (encode settings)
pub const RC_CBR: u8 = 0;
pub const RC_CQP: u8 = 1;
/// "Keep current" sentinel, the rate-control counterpart of [`CODEC_KEEP`].
///
/// A settings message says four things at once, and until this existed there
/// was no way to say only one of them: a client wanting a different bit depth
/// had to name a rate control mode and a value too, which in Auto mode means
/// overruling the controller that owns the bitrate.
pub const RC_KEEP: u8 = 0xFF;
// Bit-depth (encode settings)
pub const DEPTH_8: u8 = 0;
@@ -208,6 +286,207 @@ pub fn encode_encode_settings(
buf.push(bit_depth);
}
/// A settings payload that changes the bitrate and nothing else.
///
/// Six bytes rather than seven: the bit-depth byte is *omitted*, which
/// [`decode_encode_settings`] reports as `None`. That matters at the far end,
/// where a change carrying a depth has to be treated as a possible depth change
/// and rebuild the video session -- and a rebuild costs a keyframe. A controller
/// adjusting the bitrate every second must not do that, so it says nothing it
/// does not mean: keep the codec, keep the depth, this bitrate.
pub fn encode_bitrate_only(buf: &mut Vec<u8>, kbps: u32) {
buf.reserve(6);
buf.push(CODEC_KEEP);
buf.push(RC_CBR);
buf.extend_from_slice(&kbps.to_le_bytes());
}
// ── What colour the compositor was told a surface is ────────────────────
/// SDR: BT.709 primaries, sRGB transfer.
pub const SURFACE_COLOR_SRGB: u8 = 0;
/// HDR10: BT.2020 primaries, PQ transfer.
pub const SURFACE_COLOR_BT2020_PQ: u8 = 1;
/// What a Wayland client declared about its surface's colour.
///
/// Capture normally reads the colour space from the game's Vulkan swapchain,
/// and that is the right source when the swapchain names one. It does not
/// always: `VK_COLOR_SPACE_PASS_THROUGH_EXT` means "do not convert my values"
/// and carries no colour information at all, while the surface's real colour
/// space is declared separately, over `wp_color_manager_v1`, to the
/// compositor. A Windows title turning on HDR through wine arrives exactly
/// that way -- the pixels are BT.2020 PQ and the swapchain says nothing.
///
/// So the compositor, which is told, passes it to capture, which is not.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct SurfaceColor {
/// One of the `SURFACE_COLOR_*` constants.
pub space: u8,
/// Mastering metadata, as the client gave it. Zero where it said nothing.
///
/// Carried now and not yet applied: it belongs in the stream's own
/// metadata, and sending it from the start means that can be wired up
/// without a second protocol change and a second pin.
pub max_cll: u32,
pub max_fall: u32,
pub min_luminance: u32,
pub max_luminance: u32,
}
/// Encode a surface colour declaration:
/// `[1B space] [4B max_cll] [4B max_fall] [4B min_lum] [4B max_lum]`, LE.
pub fn encode_surface_color(buf: &mut Vec<u8>, colour: &SurfaceColor) {
buf.reserve(17);
buf.push(colour.space);
buf.extend_from_slice(&colour.max_cll.to_le_bytes());
buf.extend_from_slice(&colour.max_fall.to_le_bytes());
buf.extend_from_slice(&colour.min_luminance.to_le_bytes());
buf.extend_from_slice(&colour.max_luminance.to_le_bytes());
}
/// Decode one. `None` when the payload is too short to be one.
pub fn decode_surface_color(payload: &[u8]) -> Option<SurfaceColor> {
if payload.len() < 17 {
return None;
}
let u32_at =
|i: usize| u32::from_le_bytes([payload[i], payload[i + 1], payload[i + 2], payload[i + 3]]);
Some(SurfaceColor {
space: payload[0],
max_cll: u32_at(1),
max_fall: u32_at(5),
min_luminance: u32_at(9),
max_luminance: u32_at(13),
})
}
// ── What the client can decode ──────────────────────────────────────────
/// The codec and depth combinations a client can decode, as one bitmask.
///
/// A host that picks something the far end cannot decode produces a black
/// screen and no error, so it needs the client's whole set rather than its
/// favourite: knowing only "this one prefers AV1" leaves nowhere to fall back
/// to when the host cannot encode AV1 either.
///
/// Sent once, on connect, before any picture. That is early enough that the
/// host never sends a codec the client cannot read, which reacting to the
/// first frame could not manage.
///
/// One bit per pair, at `codec * 2 + depth`, so the layout follows from the
/// codec ids rather than from a table that can disagree with them.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct ClientCaps(u16);
/// Best first. The same order the host uses to pick among its own encoders,
/// stated once so the two cannot drift apart.
pub const CODEC_PREFERENCE: [u8; 3] = [CODEC_AV1, CODEC_H265, CODEC_H264];
impl ClientCaps {
/// Nothing supported. What a client that never spoke is assumed to have,
/// which is why [`Self::best`] treats an empty set as "no opinion" rather
/// than as "decodes nothing".
pub fn empty() -> Self {
Self(0)
}
pub fn from_bits(bits: u16) -> Self {
Self(bits)
}
pub fn bits(self) -> u16 {
self.0
}
pub fn is_empty(self) -> bool {
self.0 == 0
}
fn bit(codec: u8, depth: u8) -> Option<u16> {
// `CODEC_KEEP` and the audio codec have no place in a video capability
// set, and shifting by them would be nonsense rather than a small
// error.
if !matches!(codec, CODEC_H264 | CODEC_H265 | CODEC_AV1) {
return None;
}
if !matches!(depth, DEPTH_8 | DEPTH_10) {
return None;
}
Some(1u16 << (codec * 2 + depth))
}
/// Add one pair. Unknown codecs and depths are ignored rather than
/// panicking: this is built from what a device probe reported.
#[must_use]
pub fn with(mut self, codec: u8, depth: u8) -> Self {
if let Some(bit) = Self::bit(codec, depth) {
self.0 |= bit;
}
self
}
pub fn supports(self, codec: u8, depth: u8) -> bool {
Self::bit(codec, depth).is_some_and(|bit| self.0 & bit != 0)
}
/// Whether this set can decode `codec` at any depth.
pub fn supports_codec(self, codec: u8) -> bool {
self.supports(codec, DEPTH_8) || self.supports(codec, DEPTH_10)
}
/// The best codec and depth both ends can manage.
///
/// Walks [`CODEC_PREFERENCE`], takes the first codec present in both sets,
/// and within it prefers ten bits -- deeper coefficients carry less
/// rounding error through the transform, so it is usually a small win on
/// efficiency rather than a trade against it.
///
/// `None` when nothing overlaps, which is a real possibility rather than a
/// theoretical one: an old client that sends no capabilities at all reads
/// as empty. The caller keeps whatever it was already doing.
pub fn best(self, host: Self) -> Option<(u8, u8)> {
if self.is_empty() || host.is_empty() {
return None;
}
for codec in CODEC_PREFERENCE {
for depth in [DEPTH_10, DEPTH_8] {
if self.supports(codec, depth) && host.supports(codec, depth) {
return Some((codec, depth));
}
}
}
None
}
}
/// Encode a capability set: `[2B bits LE]`.
pub fn encode_client_caps(buf: &mut Vec<u8>, caps: ClientCaps) {
buf.extend_from_slice(&caps.bits().to_le_bytes());
}
/// Decode a capability set. `None` when the payload is too short to be one.
pub fn decode_client_caps(payload: &[u8]) -> Option<ClientCaps> {
if payload.len() < 2 {
return None;
}
Some(ClientCaps::from_bits(u16::from_le_bytes([
payload[0], payload[1],
])))
}
/// A settings payload that changes the bit depth and nothing else.
///
/// What a client sends once, on connect, to say what it can actually decode.
/// The codec and the rate control are both left alone, so this is safe to send
/// in Auto mode, where the controller owns the bitrate.
///
/// Seven bytes, because the depth byte is the seventh: see
/// [`encode_bitrate_only`] for why its absence means something.
pub fn encode_depth_only(buf: &mut Vec<u8>, bit_depth: u8) {
encode_encode_settings(buf, CODEC_KEEP, RC_KEEP, 0, bit_depth);
}
/// Decode an encode-settings payload. Returns `(codec_id, rate_control_mode, value, bit_depth)`.
/// bit_depth is None for 6-byte (old client) payloads, Some(n) for 7+ byte payloads.
pub fn decode_encode_settings(payload: &[u8]) -> Option<(u8, u8, u32, Option<u8>)> {
@@ -224,3 +503,439 @@ pub fn decode_encode_settings(payload: &[u8]) -> Option<(u8, u8, u32, Option<u8>
};
Some((codec_id, rc, value, depth))
}
// ── Receiver report ─────────────────────────────────────────────
/// What one second looked like from the receiving end.
///
/// Counts are per-second deltas, not totals: a controller wants to know what is
/// happening now, and a total makes every reading depend on how long the session
/// has been running.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct ReceiverReport {
/// Video bits per second actually reassembled and released to the decoder.
///
/// Not what was sent, and not what arrived -- what *completed*. A frame
/// missing one fragment contributes nothing here, which is right: it
/// contributed nothing to the picture either. When the sender is saturating
/// the path this is the measured capacity of it.
pub goodput_bps: u64,
/// Frames released to the decoder.
pub released: u32,
/// Frames that started arriving and never completed.
pub incomplete: u32,
/// Frames no fragment of which ever arrived.
pub never_arrived: u32,
/// The receiver's own round-trip estimate, in milliseconds.
pub rtt_ms: u32,
}
impl ReceiverReport {
/// The fraction of frames that did not make it, in `0.0..=1.0`.
///
/// `None` when no frames were accounted for at all, which is not the same
/// as no loss -- a second in which nothing was sent and a second in which
/// nothing arrived look identical here, and only the caller knows which it
/// is expecting.
pub fn loss(&self) -> Option<f32> {
let total = self.released + self.incomplete + self.never_arrived;
if total == 0 {
return None;
}
Some((self.incomplete + self.never_arrived) as f32 / total as f32)
}
}
/// `[8B goodput_bps][4B released][4B incomplete][4B never_arrived][4B rtt_ms]`,
/// all little-endian.
pub const RECEIVER_REPORT_LEN: usize = 24;
pub fn encode_receiver_report(buf: &mut Vec<u8>, report: &ReceiverReport) {
buf.reserve(RECEIVER_REPORT_LEN);
buf.extend_from_slice(&report.goodput_bps.to_le_bytes());
buf.extend_from_slice(&report.released.to_le_bytes());
buf.extend_from_slice(&report.incomplete.to_le_bytes());
buf.extend_from_slice(&report.never_arrived.to_le_bytes());
buf.extend_from_slice(&report.rtt_ms.to_le_bytes());
}
/// Decode a receiver report. `None` when the payload is short.
///
/// A payload *longer* than expected is accepted and its tail ignored, so a newer
/// client that appends a field still reports usefully to an older hub.
pub fn decode_receiver_report(payload: &[u8]) -> Option<ReceiverReport> {
if payload.len() < RECEIVER_REPORT_LEN {
return None;
}
let u32_at = |o: usize| u32::from_le_bytes(payload[o..o + 4].try_into().unwrap());
Some(ReceiverReport {
goodput_bps: u64::from_le_bytes(payload[0..8].try_into().unwrap()),
released: u32_at(8),
incomplete: u32_at(12),
never_arrived: u32_at(16),
rtt_ms: u32_at(20),
})
}
// ── Control mode ────────────────────────────────────────────────
/// Who is choosing the bitrate.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ControlMode {
/// The hub's controller decides, within the ceiling.
#[default]
Auto,
/// A person decided, and the controller stands down until told otherwise.
///
/// Kept because it is how this class of bug gets diagnosed at all: the
/// original "the bitrate is already lowered" report was wrong, and the only
/// way anyone established that was by setting one by hand and watching the
/// picture come back.
Manual,
}
pub const CONTROL_MODE_AUTO: u8 = 0;
pub const CONTROL_MODE_MANUAL: u8 = 1;
/// `[1B mode][4B ceiling_kbps LE]`. A ceiling of 0 means "no opinion, keep
/// whatever the hub was given".
pub const CONTROL_MODE_LEN: usize = 5;
pub fn encode_control_mode(buf: &mut Vec<u8>, mode: ControlMode, ceiling_kbps: u32) {
buf.reserve(CONTROL_MODE_LEN);
buf.push(match mode {
ControlMode::Auto => CONTROL_MODE_AUTO,
ControlMode::Manual => CONTROL_MODE_MANUAL,
});
buf.extend_from_slice(&ceiling_kbps.to_le_bytes());
}
/// Returns `(mode, ceiling_kbps)`; the ceiling is `None` when it was left at 0.
pub fn decode_control_mode(payload: &[u8]) -> Option<(ControlMode, Option<u32>)> {
if payload.len() < CONTROL_MODE_LEN {
return None;
}
let mode = match payload[0] {
CONTROL_MODE_AUTO => ControlMode::Auto,
CONTROL_MODE_MANUAL => ControlMode::Manual,
// An unknown mode is not a reason to stop controlling the bitrate, and
// guessing "manual" would silently disable the controller.
_ => return None,
};
let ceiling = u32::from_le_bytes(payload[1..5].try_into().unwrap());
Some((mode, (ceiling != 0).then_some(ceiling)))
}
#[cfg(test)]
mod media_control_tests {
use super::*;
fn report() -> ReceiverReport {
ReceiverReport {
goodput_bps: 2_850_000,
released: 47,
incomplete: 12,
never_arrived: 1,
rtt_ms: 182,
}
}
#[test]
fn a_report_survives_the_wire() {
let mut buf = Vec::new();
encode_receiver_report(&mut buf, &report());
assert_eq!(buf.len(), RECEIVER_REPORT_LEN);
assert_eq!(decode_receiver_report(&buf), Some(report()));
}
#[test]
fn a_short_report_is_refused_rather_than_guessed() {
let mut buf = Vec::new();
encode_receiver_report(&mut buf, &report());
for n in 0..RECEIVER_REPORT_LEN {
assert_eq!(decode_receiver_report(&buf[..n]), None, "{n} bytes");
}
}
#[test]
fn a_longer_report_is_read_and_its_tail_ignored() {
// So a newer client that appends a field still reports usefully to a
// hub that predates it.
let mut buf = Vec::new();
encode_receiver_report(&mut buf, &report());
buf.extend_from_slice(&[0xAA; 8]);
assert_eq!(decode_receiver_report(&buf), Some(report()));
}
#[test]
fn loss_counts_every_frame_that_did_not_arrive_whole() {
// An incomplete frame is a lost frame. It cost bandwidth and produced no
// picture, which is worse than never having been sent.
let r = ReceiverReport {
released: 90,
incomplete: 8,
never_arrived: 2,
..Default::default()
};
assert_eq!(r.loss(), Some(0.1));
}
#[test]
fn a_silent_second_has_no_loss_figure() {
// Nothing sent and nothing arrived look identical from here. Reporting
// 0% would tell a controller the path is healthy; reporting 100% would
// tell it to collapse the bitrate. Neither is known, so neither is said.
assert_eq!(ReceiverReport::default().loss(), None);
}
#[test]
fn total_loss_is_reported_as_total() {
let r = ReceiverReport {
released: 0,
incomplete: 46,
never_arrived: 14,
..Default::default()
};
assert_eq!(r.loss(), Some(1.0));
}
#[test]
fn a_control_mode_survives_the_wire() {
for (mode, ceiling) in [
(ControlMode::Auto, 8_000u32),
(ControlMode::Manual, 1_000),
(ControlMode::Auto, 0),
] {
let mut buf = Vec::new();
encode_control_mode(&mut buf, mode, ceiling);
assert_eq!(buf.len(), CONTROL_MODE_LEN);
assert_eq!(
decode_control_mode(&buf),
Some((mode, (ceiling != 0).then_some(ceiling))),
);
}
}
#[test]
fn an_unknown_mode_is_refused_rather_than_defaulted() {
// Defaulting to manual would silently switch the controller off, which
// is the failure this whole change exists to remove.
let mut buf = vec![0x7F];
buf.extend_from_slice(&8_000u32.to_le_bytes());
assert_eq!(decode_control_mode(&buf), None);
}
#[test]
fn a_surface_colour_survives_the_wire() {
let colour = SurfaceColor {
space: SURFACE_COLOR_BT2020_PQ,
max_cll: 1000,
max_fall: 400,
min_luminance: 0,
max_luminance: 1000,
};
let mut buf = Vec::new();
encode_surface_color(&mut buf, &colour);
assert_eq!(buf.len(), 17);
assert_eq!(decode_surface_color(&buf), Some(colour));
}
/// A truncated payload is not a surface that is suddenly SDR. Reading one
/// as though it were would turn a dropped byte into a wrong picture.
#[test]
fn a_short_surface_colour_is_not_read() {
let mut buf = Vec::new();
encode_surface_color(&mut buf, &SurfaceColor::default());
for len in 0..17 {
assert_eq!(decode_surface_color(&buf[..len]), None, "len {len}");
}
}
#[test]
fn every_message_type_is_its_own_number() {
// Every type byte that travels on a stream, as `(name, value)`. Listed
// by hand because the point is to catch a new one colliding with an
// existing one, and anything derived from the constants would agree
// with them by construction.
//
// `MSG_CLIENT_CAPS` was 0x13 when it was added, which is
// `MSG_RECEIVER_REPORT`. The hub matches on the type byte and the caps
// arm came first, so every receiver report would have been read as
// capabilities - taking away the only measurement the bitrate
// controller has, silently, on a message sent once per connection.
let types = [
("MSG_DATA", MSG_DATA),
("MSG_IDR_REQUEST", MSG_IDR_REQUEST),
("MSG_ENCODE_SETTINGS", MSG_ENCODE_SETTINGS),
("MSG_CLIENT_CAPS", MSG_CLIENT_CAPS),
("MSG_SURFACE_COLOR", MSG_SURFACE_COLOR),
("MSG_RECEIVER_REPORT", MSG_RECEIVER_REPORT),
("MSG_CONTROL_MODE", MSG_CONTROL_MODE),
("MSG_INPUT_BATCH", MSG_INPUT_BATCH),
];
for (i, (name, value)) in types.iter().enumerate() {
for (other_name, other_value) in &types[i + 1..] {
assert_ne!(
value, other_value,
"{name} and {other_name} are both {value:#04x}"
);
}
}
}
#[test]
fn every_codec_and_depth_has_its_own_bit() {
let all = [CODEC_H264, CODEC_H265, CODEC_AV1]
.into_iter()
.flat_map(|c| [DEPTH_8, DEPTH_10].map(move |d| (c, d)));
let mut seen = Vec::new();
for (codec, depth) in all {
let caps = ClientCaps::empty().with(codec, depth);
assert!(caps.supports(codec, depth));
assert!(!seen.contains(&caps.bits()), "{codec}/{depth} collides");
seen.push(caps.bits());
}
}
/// Nothing outside the video codecs belongs in a capability set, and a
/// shift by `CODEC_KEEP` would be nonsense rather than a small error.
#[test]
fn nonsense_pairs_are_ignored_rather_than_stored() {
let caps = ClientCaps::empty()
.with(CODEC_KEEP, DEPTH_8)
.with(CODEC_OPUS, DEPTH_8)
.with(CODEC_AV1, 7);
assert!(caps.is_empty());
assert!(!caps.supports(CODEC_KEEP, DEPTH_8));
}
fn host_all() -> ClientCaps {
ClientCaps::empty()
.with(CODEC_AV1, DEPTH_8)
.with(CODEC_AV1, DEPTH_10)
.with(CODEC_H265, DEPTH_8)
.with(CODEC_H265, DEPTH_10)
.with(CODEC_H264, DEPTH_8)
}
#[test]
fn the_best_shared_codec_wins_at_the_deeper_depth() {
assert_eq!(host_all().best(host_all()), Some((CODEC_AV1, DEPTH_10)));
}
/// The case this exists for: a client with no AV1 decoder must not be sent
/// AV1 just because the host prefers it.
#[test]
fn a_client_without_av1_gets_h265() {
let client = ClientCaps::empty()
.with(CODEC_H265, DEPTH_8)
.with(CODEC_H265, DEPTH_10)
.with(CODEC_H264, DEPTH_8);
assert_eq!(client.best(host_all()), Some((CODEC_H265, DEPTH_10)));
}
#[test]
fn a_client_with_only_h264_gets_h264() {
let client = ClientCaps::empty().with(CODEC_H264, DEPTH_8);
assert_eq!(client.best(host_all()), Some((CODEC_H264, DEPTH_8)));
}
/// Ten bits is preferred, not required: a client that decodes H.265 at
/// eight bits only still gets H.265 rather than being pushed to H.264.
#[test]
fn eight_bit_is_taken_when_that_is_all_there_is() {
let client = ClientCaps::empty()
.with(CODEC_H265, DEPTH_8)
.with(CODEC_H264, DEPTH_8);
assert_eq!(client.best(host_all()), Some((CODEC_H265, DEPTH_8)));
}
/// A host that can only encode AV1 and a client that cannot decode it
/// share nothing. The caller keeps what it was doing rather than picking
/// something neither end asked for.
#[test]
fn no_overlap_is_no_answer() {
let host = ClientCaps::empty().with(CODEC_AV1, DEPTH_8);
let client = ClientCaps::empty().with(CODEC_H264, DEPTH_8);
assert_eq!(client.best(host), None);
}
/// A client that never sent capabilities reads as empty, which must mean
/// "said nothing" and not "decodes nothing".
#[test]
fn silence_is_not_an_answer_either() {
assert_eq!(ClientCaps::empty().best(host_all()), None);
assert_eq!(host_all().best(ClientCaps::empty()), None);
}
#[test]
fn capabilities_survive_the_wire() {
let caps = ClientCaps::empty()
.with(CODEC_AV1, DEPTH_10)
.with(CODEC_H264, DEPTH_8);
let mut buf = Vec::new();
encode_client_caps(&mut buf, caps);
assert_eq!(decode_client_caps(&buf), Some(caps));
assert_eq!(decode_client_caps(&buf[..1]), None, "too short to read");
}
/// The host walks its own encoders in this order; stating it once is what
/// keeps the two ends agreeing about what "best" means.
#[test]
fn the_preference_order_is_av1_first() {
assert_eq!(CODEC_PREFERENCE, [CODEC_AV1, CODEC_H265, CODEC_H264]);
}
#[test]
fn a_depth_only_change_touches_nothing_else() {
let mut buf = Vec::new();
encode_depth_only(&mut buf, DEPTH_10);
let (codec, rc, value, depth) = decode_encode_settings(&buf).expect("readable");
assert_eq!(codec, CODEC_KEEP, "the codec is the host's business");
assert_eq!(rc, RC_KEEP, "the controller keeps the bitrate it chose");
assert_eq!(value, 0, "and there is no value to read");
assert_eq!(depth, Some(DEPTH_10), "the depth is the whole message");
}
/// The two sentinels have to be distinguishable from real values, or a
/// "keep this" reads as a request for something.
#[test]
fn the_keep_sentinels_are_not_real_settings() {
assert_ne!(RC_KEEP, RC_CBR);
assert_ne!(RC_KEEP, RC_CQP);
assert_ne!(CODEC_KEEP, CODEC_H264);
assert_ne!(CODEC_KEEP, CODEC_H265);
assert_ne!(CODEC_KEEP, CODEC_AV1);
}
#[test]
fn a_bitrate_only_change_carries_no_depth_and_no_codec() {
// The far end rebuilds its video session -- and spends a keyframe -- for
// anything that might be a codec or depth change. A controller nudging
// the bitrate every second must say neither.
let mut buf = Vec::new();
encode_bitrate_only(&mut buf, 2_500);
let (codec, rc, value, depth) = decode_encode_settings(&buf).expect("readable");
assert_eq!(codec, CODEC_KEEP);
assert_eq!(rc, RC_CBR);
assert_eq!(value, 2_500);
assert_eq!(depth, None, "a depth byte would force a rebuild");
}
#[test]
fn the_new_message_types_do_not_collide() {
let all = [
MSG_DATA,
MSG_IDR_REQUEST,
MSG_ENCODE_SETTINGS,
MSG_RECEIVER_REPORT,
MSG_CONTROL_MODE,
MSG_INPUT_BATCH,
];
for (i, a) in all.iter().enumerate() {
for b in &all[i + 1..] {
assert_ne!(a, b, "two message types share a value");
}
}
}
}
crates/nesprotocol/src/lifecycle.rs
+122 -10
View File
@@ -59,7 +59,10 @@ pub const CONTROL_PORT: u32 = 7000;
/// messages, so a version-2 peer and a version-3 peer do not talk at all.
/// There is deliberately no shim: nothing is deployed, and a shim would be the
/// second definition of this wire that one shared crate exists to prevent.
pub const CONTROL_VERSION: u32 = 3;
///
/// Version 4 replaced the descriptor's `drives` with `overlays`: a build is a
/// read-only image now, and a box writes into a layer of its own over it.
pub const CONTROL_VERSION: u32 = 4;
/// The command to run, and who runs it.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -101,15 +104,29 @@ pub struct Mount {
pub ro: bool,
}
/// A block device the guest mounts, rather than a share it is handed.
/// A read-only block device with a writable one layered over it.
///
/// There are no mount options on this and that is deliberate: what the guest
/// mounts a build volume with is a property of how the volume was built --
/// journal-less ext4, `nosuid`, `nodev` -- and not something a descriptor is in
/// a position to know. A field for options was here and was never read.
/// This is how a game's install reaches a box. The build is an EROFS image
/// every box on the host shares, and the box's own writes land on an ext4 of
/// its own that is thrown away with it; the guest stacks the two with
/// overlayfs, so a game that writes into its install directory works and the
/// build under it cannot change.
///
/// Stacked in the guest rather than on the host because the host has no
/// filesystem it can do it on: the host-side equivalent was a ZFS clone, and a
/// host that needs ZFS is what this replaced.
///
/// There are no mount options on this and that is deliberate: how each layer
/// is mounted is a property of how it was built -- EROFS read-only, a
/// journal-less ext4 upper, `nosuid`, `nodev` -- and not something a
/// descriptor is in a position to know.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Drive {
pub dev: String,
pub struct Overlay {
/// The read-only build image, as the guest names the device.
pub lower: String,
/// The box's writable layer, as the guest names the device.
pub upper: String,
/// Where the stacked result lands.
pub at: String,
}
@@ -178,7 +195,40 @@ pub struct BootDescriptor {
#[serde(default)]
pub mounts: Vec<Mount>,
#[serde(default)]
pub drives: Vec<Drive>,
pub overlays: Vec<Overlay>,
/// What the box may spend on video.
///
/// Here rather than on a launch because its consumer is `neshub`, which is
/// a service and comes up with the box. Geometry went the other way for the
/// same reason: its consumer is the compositor, which is started per launch.
/// A number travels to where the thing that reads it is started.
#[serde(default)]
pub video: VideoLimits,
}
/// Ceilings on what a box's video may cost.
///
/// A struct rather than a bare number so the next video-shaped limit joins it
/// instead of arriving loose alongside it.
///
/// Note what `deny_unknown_fields` on [`BootDescriptor`] means for this: a host
/// that sends `video` to a guest too old to know the field is *refused*, not
/// quietly accepted. That is the intended direction of failure -- the
/// alternative is a box that comes up, streams, and ignores the ceiling it was
/// given, which is exactly the shape of failure this stack produces too easily.
/// The guest image is rebuilt before a host starts sending it.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct VideoLimits {
/// Ceiling on the video bitrate, in kbps.
///
/// `None` means nobody said, which is not the same as zero and is not the
/// same as unlimited. A reader that was told nothing should say so and pick
/// a conservative default of its own; a reader that treats "unsaid" as
/// "unlimited" reproduces the bug this exists to fix, where every session
/// offered 10 Mbps because nothing had ever set a number.
#[serde(skip_serializing_if = "Option::is_none", default)]
pub bitrate_kbps: Option<u32>,
}
/// How a workload ended.
@@ -390,7 +440,8 @@ mod tests {
at: "/mnt/install".into(),
ro: true,
}],
drives: Vec::new(),
overlays: Vec::new(),
video: VideoLimits::default(),
}
}
@@ -633,3 +684,64 @@ mod tests {
assert!(!on_exit.terminal);
}
}
#[cfg(test)]
mod video_limits_tests {
use super::*;
#[test]
fn a_descriptor_without_video_still_reads() {
// An older host says nothing about video. That has to keep working, and
// it has to be distinguishable from a host that said "no limit".
let d: BootDescriptor = serde_json::from_str(r#"{"mounts":[],"overlays":[]}"#).unwrap();
assert_eq!(d.video.bitrate_kbps, None);
}
#[test]
fn an_unsaid_ceiling_is_not_serialised() {
// So a host that has nothing to say produces the same bytes it always
// did, and an older guest keeps accepting it.
let d = BootDescriptor {
mounts: Vec::new(),
overlays: Vec::new(),
video: VideoLimits::default(),
};
let json = serde_json::to_string(&d).unwrap();
assert!(!json.contains("bitrate"), "{json}");
}
#[test]
fn a_ceiling_survives_the_round_trip() {
let d = BootDescriptor {
mounts: Vec::new(),
overlays: Vec::new(),
video: VideoLimits {
bitrate_kbps: Some(8_000),
},
};
let back: BootDescriptor =
serde_json::from_str(&serde_json::to_string(&d).unwrap()).unwrap();
assert_eq!(back, d);
}
#[test]
fn zero_is_not_the_same_as_unsaid() {
// A reader that conflates them cannot tell "the host wants no video" from
// "the host never mentioned it", and the second must not be read as a
// licence to send whatever it likes.
let said: BootDescriptor = serde_json::from_str(r#"{"video":{"bitrate_kbps":0}}"#).unwrap();
let unsaid: BootDescriptor = serde_json::from_str("{}").unwrap();
assert_eq!(said.video.bitrate_kbps, Some(0));
assert_eq!(unsaid.video.bitrate_kbps, None);
}
#[test]
fn an_unknown_video_field_is_refused() {
// Same reasoning as the descriptor's own `deny_unknown_fields`: a limit
// this build does not understand is one it would otherwise ignore while
// reporting success.
let r: Result<BootDescriptor, _> =
serde_json::from_str(r#"{"video":{"bitrate_kbps":8000,"fps_cap":30}}"#);
assert!(r.is_err(), "an unknown video limit must not be ignored");
}
}
crates/nesprotocol/src/stats.rs
+204 -10
View File
@@ -13,9 +13,9 @@ pub fn encode_nescope_stats(buf: &mut Vec<u8>, game_fps: u8, frame_count: u32) {
buf.extend_from_slice(&frame_count.to_le_bytes());
}
/// Hudless stats: capture FPS, encode time, dropped frames, diagnostics, capture latency.
/// nescapture stats: capture FPS, encode time, dropped frames, diagnostics, capture latency.
/// [1][capture_fps: u8][encode_avg_ms: f32 LE][dropped: u32 LE][present_attempts: u32 LE][capture_attempts: u32 LE][capture_ms: f32 LE]
pub fn encode_hudless_stats(
pub fn encode_nescapture_stats(
buf: &mut Vec<u8>,
capture_fps: u8,
encode_avg_ms: f32,
@@ -60,15 +60,98 @@ pub fn encode_hub_stats(
buf.push(audio_channels);
}
/// What the video bitrate is made of, and who chose it.
///
/// Appended to a hub stats packet rather than replacing anything, so an older
/// reader keeps working on the part it understands -- `decode_stats` already
/// guards each field on the length it needs.
///
/// **The split is the point.** One combined byte counter cannot distinguish an
/// encoder ignoring its bitrate target from a stream that is mostly keyframes,
/// and those have opposite fixes. A session overshooting its target by ten times
/// looked identical either way, which is why the cause stayed ambiguous for
/// weeks.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct VideoBreakdown {
/// Keyframe bits per second, measured over the last second.
pub key_bps: u32,
/// Everything else, measured the same way.
pub delta_bps: u32,
/// Keyframes in the last second.
pub keyframes: u8,
/// What the controller is asking the encoder for.
pub target_kbps: u32,
/// The ceiling it is choosing within, after any client lowered it.
pub ceiling_kbps: u32,
/// How much the box's own pipeline delay *varied* over the last second, in
/// milliseconds, at the median and the 95th percentile.
///
/// Variation rather than absolute cost, and necessarily so: the encoder
/// stamps a frame with milliseconds since its own start, not since any
/// epoch, so the difference to wall clock holds an unknown constant even
/// though both run on the same machine. It is also the comparable
/// quantity -- the client measures the same thing about the total arrival
/// delay, with the same code, so the gap between the two is what the
/// network added.
///
/// The point is attribution: a client seeing late frames cannot otherwise
/// tell a stalled encoder from a jittery path, and buffering against the
/// first is latency spent hiding a fault that should be fixed instead.
pub pipeline_p50_ms: u16,
pub pipeline_p95_ms: u16,
/// The worst single frame in that second.
pub pipeline_max_ms: u16,
/// The ceiling the box itself was given, which no client may exceed.
///
/// Separate from `ceiling_kbps` because a client that lowers the ceiling
/// would otherwise have nothing left to raise it against: the only number
/// it can see is the one it just lowered. A control that can be turned down
/// and not back up is worse than no control.
pub box_ceiling_kbps: u32,
/// Why the target is what it is; see the hub's control module.
pub reason: u8,
/// 0 when the controller is deciding, 1 when a person set it by hand.
pub manual: u8,
/// How far behind the send queue is, in milliseconds of video.
///
/// Bytes handed to the transport and not yet gone, over the rate they are
/// leaving at. This is the number that was invisible during the session
/// where every frame arrived, nothing was lost, and the picture was still
/// eight seconds old: loss cannot show a queue, only its overflow. Anything
/// but near-zero here means latency is being spent on backlog.
pub backlog_ms: u16,
}
/// `[4B key_bps][4B delta_bps][1B keyframes][4B target][4B ceiling][1B reason]
/// [1B manual][4B box_ceiling][2B pipeline_p50][2B pipeline_p95][2B pipeline_max]
/// [2B backlog_ms]`
pub const VIDEO_BREAKDOWN_LEN: usize = 31;
pub fn encode_video_breakdown(buf: &mut Vec<u8>, b: &VideoBreakdown) {
buf.reserve(VIDEO_BREAKDOWN_LEN);
buf.extend_from_slice(&b.key_bps.to_le_bytes());
buf.extend_from_slice(&b.delta_bps.to_le_bytes());
buf.push(b.keyframes);
buf.extend_from_slice(&b.target_kbps.to_le_bytes());
buf.extend_from_slice(&b.ceiling_kbps.to_le_bytes());
buf.push(b.reason);
buf.push(b.manual);
buf.extend_from_slice(&b.box_ceiling_kbps.to_le_bytes());
buf.extend_from_slice(&b.pipeline_p50_ms.to_le_bytes());
buf.extend_from_slice(&b.pipeline_p95_ms.to_le_bytes());
buf.extend_from_slice(&b.pipeline_max_ms.to_le_bytes());
buf.extend_from_slice(&b.backlog_ms.to_le_bytes());
}
/// Decoded stats from any source.
#[derive(Debug, Clone, Default)]
pub struct PipelineStats {
pub nescope_fps: u8,
pub nescope_frames: u32,
pub hudless_fps: u8,
pub hudless_encode_ms: f32,
pub hudless_capture_ms: f32,
pub hudless_dropped: u32,
pub nescapture_fps: u8,
pub nescapture_encode_ms: f32,
pub nescapture_capture_ms: f32,
pub nescapture_dropped: u32,
pub hub_clients: u8,
pub hub_video_mb: u32,
pub hub_relay_ms: f32,
@@ -76,6 +159,8 @@ pub struct PipelineStats {
pub capture_attempts: u32,
pub audio_bitrate_kbps: u32,
pub audio_channels: u8,
/// `None` from a hub that predates the breakdown.
pub video: Option<VideoBreakdown>,
}
/// Try to decode a single stats packet. The `msg_type` is the frame-level
@@ -87,13 +172,13 @@ pub fn decode_stats(msg_type: u8, data: &[u8], stats: &mut PipelineStats) {
stats.nescope_frames = u32::from_le_bytes([data[1], data[2], data[3], data[4]]);
}
STATS_HUDLESS if data.len() >= 17 => {
stats.hudless_fps = data[0];
stats.hudless_encode_ms = f32::from_le_bytes([data[1], data[2], data[3], data[4]]);
stats.hudless_dropped = u32::from_le_bytes([data[5], data[6], data[7], data[8]]);
stats.nescapture_fps = data[0];
stats.nescapture_encode_ms = f32::from_le_bytes([data[1], data[2], data[3], data[4]]);
stats.nescapture_dropped = u32::from_le_bytes([data[5], data[6], data[7], data[8]]);
stats.present_attempts = u32::from_le_bytes([data[9], data[10], data[11], data[12]]);
stats.capture_attempts = u32::from_le_bytes([data[13], data[14], data[15], data[16]]);
if data.len() >= 21 {
stats.hudless_capture_ms =
stats.nescapture_capture_ms =
f32::from_le_bytes([data[17], data[18], data[19], data[20]]);
}
}
@@ -110,7 +195,116 @@ pub fn decode_stats(msg_type: u8, data: &[u8], stats: &mut PipelineStats) {
if data.len() >= 14 {
stats.audio_channels = data[13];
}
if data.len() >= 14 + VIDEO_BREAKDOWN_LEN {
let d = &data[14..];
let u32_at = |o: usize| u32::from_le_bytes(d[o..o + 4].try_into().unwrap());
stats.video = Some(VideoBreakdown {
key_bps: u32_at(0),
delta_bps: u32_at(4),
keyframes: d[8],
target_kbps: u32_at(9),
ceiling_kbps: u32_at(13),
reason: d[17],
manual: d[18],
box_ceiling_kbps: u32_at(19),
pipeline_p50_ms: u16::from_le_bytes([d[23], d[24]]),
pipeline_p95_ms: u16::from_le_bytes([d[25], d[26]]),
pipeline_max_ms: u16::from_le_bytes([d[27], d[28]]),
backlog_ms: u16::from_le_bytes([d[29], d[30]]),
});
}
}
_ => {}
}
}
#[cfg(test)]
mod breakdown_tests {
use super::*;
fn breakdown() -> VideoBreakdown {
VideoBreakdown {
key_bps: 3_200_000,
delta_bps: 6_800_000,
keyframes: 2,
target_kbps: 6_000,
ceiling_kbps: 8_000,
reason: 1,
manual: 0,
box_ceiling_kbps: 8_000,
pipeline_p50_ms: 9,
pipeline_p95_ms: 24,
pipeline_max_ms: 61,
backlog_ms: 40,
}
}
fn hub_packet(with_breakdown: bool) -> Vec<u8> {
let mut buf = Vec::new();
encode_hub_stats(&mut buf, 1, 10_000_000, 0.0, 128, 2);
if with_breakdown {
encode_video_breakdown(&mut buf, &breakdown());
}
buf
}
#[test]
fn a_breakdown_survives_the_wire() {
let packet = hub_packet(true);
let mut stats = PipelineStats::default();
decode_stats(STATS_HUB, &packet[1..], &mut stats);
assert_eq!(stats.video, Some(breakdown()));
// The fields that were always there still read correctly beside it.
assert_eq!(stats.hub_clients, 1);
assert_eq!(stats.audio_bitrate_kbps, 128);
assert_eq!(stats.audio_channels, 2);
}
#[test]
fn a_hub_without_the_breakdown_still_reads() {
// The reason this is appended rather than folded into the layout: a hub
// that predates it keeps working, and says so by omission rather than by
// reporting zeroes that look like a stream carrying nothing.
let packet = hub_packet(false);
let mut stats = PipelineStats::default();
decode_stats(STATS_HUB, &packet[1..], &mut stats);
assert_eq!(stats.video, None);
assert_eq!(stats.hub_clients, 1);
assert_eq!(stats.audio_channels, 2);
}
#[test]
fn a_truncated_breakdown_is_left_out_rather_than_half_read() {
let full = hub_packet(true);
for n in 15..full.len() - 1 {
let mut stats = PipelineStats::default();
decode_stats(STATS_HUB, &full[1..n], &mut stats);
assert_eq!(stats.video, None, "{n} bytes produced a partial breakdown");
}
}
#[test]
fn the_split_distinguishes_the_two_ways_a_stream_overshoots() {
// The whole reason for the split. Same total, opposite causes: an
// encoder ignoring its target, and a stream that is nearly all
// keyframes. One counter cannot tell them apart.
let ignoring_target = VideoBreakdown {
key_bps: 200_000,
delta_bps: 9_800_000,
keyframes: 1,
..breakdown()
};
let keyframe_storm = VideoBreakdown {
key_bps: 9_000_000,
delta_bps: 1_000_000,
keyframes: 30,
..breakdown()
};
assert_eq!(
ignoring_target.key_bps + ignoring_target.delta_bps,
keyframe_storm.key_bps + keyframe_storm.delta_bps,
);
assert!(ignoring_target.delta_bps > ignoring_target.key_bps);
assert!(keyframe_storm.key_bps > keyframe_storm.delta_bps);
}
}
docs/media-transport.md
+208
View File
@@ -0,0 +1,208 @@
# The media transport, as it actually behaves
Media rides one QUIC connection: video delta frames and audio as datagrams,
keyframes as short-lived unidirectional streams, cursor and stats and input as
streams of their own. Several things about that arrangement are not what they
look like, and we designed against the wrong model twice before reading the
source.
This file records what the transport *does*, with somewhere to check each
claim, and what follows from it. It is deliberately about behaviour rather than
about our code, because our code changes and this does not — except when the
dependency does, which is why the version is named.
Read against **iroh 1.1, which uses `noq` and `noq-proto` 1.3 — a quinn fork,
not quinn itself.** Paths below are inside `noq-proto` unless stated. Reading
upstream quinn instead is a mistake worth naming, because we made it: the
behaviour turned out to agree on every point here, but "turned out to" is
precisely the standard this file exists to replace. If the iroh or noq version
moves, re-read before trusting any of this.
## The rule
**A path is the unit of congestion, not a stream and not a connection.**
Everything else here follows from that one sentence, and almost every wrong
assumption we made came from forgetting some part of it.
## Congestion is per path, and shared by everything on that path
Each path carries its own congestion controller and its own pacer
(`connection/paths.rs`, `PathData::congestion` and `PathData::pacing_delay`),
and `poll_transmit` consults both for the path it is about to send on
(`connection/mod.rs`, in the transmit blocking checks). Within a path, every
stream and every datagram shares them, whatever the packet ends up carrying.
The "per path" part is not pedantry here: a connection commonly holds a relay
path and a direct path at once, they have different windows and different
round-trip times, and only the *selected* one describes where media is
currently going. Reading connection-level aggregates instead will average two
unrelated network conditions together.
So moving keyframes onto their own streams — which we did, and which was right
— changed the *delivery guarantee* and nothing else. A lost fragment no longer
destroys a keyframe, because QUIC retransmits stream data. But that keyframe
still competes for the same congestion window as everything else on the path,
and still queues in the same scheduler. Stream-versus-datagram is a reliability
distinction, not an isolation one.
The corollary is worth stating because it is tempting and wrong: **opening a
second connection does not create bandwidth.** At a 3 Mbps bottleneck a 150 KB
keyframe occupies about 400 ms of link time however it is carried. A second
connection turns strict queueing into competition between two flows over the
same bottleneck, which is sometimes what you want — but it is a change in
*fairness*, not in capacity, and it costs a second NAT traversal, a second
handshake and a second set of paths to manage.
## Datagrams are written before stream data
`populate_packet` fills each packet in a fixed order, and DATAGRAM frames come
before STREAM frames. Whatever datagrams are queued take their space first;
stream data gets the remainder.
Two consequences, and they point in opposite directions:
**Delta frames cannot be starved by keyframes.** A large keyframe simply
stretches out over many packets while deltas keep flowing beside it. This is
the behaviour we nearly built by hand before checking.
**Keyframes can be starved by delta frames.** On a saturated link a steady
stream of datagrams fills every packet, and the keyframe on its stream gets
leftovers. If the client is waiting on that keyframe to resynchronise, the
frames starving it are the very frames it cannot decode without it. That is a
feedback loop with no bottom, and it is what produced "26 keyframe fallbacks,
19 IDR requests" in the field: the recovery frame could not get out past the
frames that needed it to arrive first.
The fix is not to reorder the transport. It is to stop sending data that
depends on data the receiver does not have — see `dgram::ResyncGate`.
## The send path has no backpressure signal
`send_datagram` calls `datagrams().send(data, true)` (`noq/src/connection.rs`),
and that `true` is a `drop` flag: the queue makes room by **evicting the oldest
queued datagrams** via `make_space_for` and then returns `Ok`
(`connection/datagrams.rs`). It errors only for `TooLarge`, `UnsupportedByPeer`,
`Disabled`, and `ConnectionLost` — that is, for a datagram that was malformed
or a connection that is already gone, never for one the path could not carry.
(The same function with `drop: false` returns `Blocked` instead and is what
`send_datagram_wait` uses. That would give backpressure, at the cost of
prioritising old frames over current ones, which for live media is the wrong
trade.)
A sender overrunning the path is therefore told nothing at all. Any counter
fed by that return value cannot be non-zero however badly things are going,
which is exactly how a field report came to contain "encoder perfectly healthy,
zero frames dropped" beside a client receiving almost nothing. `send_datagram`
is still the right call for live media — waiting for buffer space prioritises
old frames over current ones — but its silence has to be designed around
rather than read as good news.
What *can* be read: `datagram_send_buffer_space()` for remaining room, and the
selected path's `stats()` for `cwnd`, `rtt` and `lost_packets`. Note "selected":
a connection commonly holds a relay path and a direct path at once, and only
the selected one describes where media is going.
## quinn already paces
`poll_transmit` blocks on a token-bucket pacer (`connection/pacing.rs`, reached
through `PathData::pacing_delay`) sized from that path's congestion window and
round-trip time, before every packet. Handing 150 fragments over in a tight
loop does *not* put 150 packets on the wire at once.
Application-level pacing on top of this is redundant, and worse than
redundant: it would spread transmission according to our guess at the available
rate, while the pacer underneath is using the measured window and RTT. If the
application is producing faster than the pacer drains, the answer is to produce
less — which is the bitrate controller's job — not to add a second pacer.
## Loss is a lagging indicator; the queue is the leading one
A path that is overrun does not necessarily drop anything. It can simply be
slower than the offer, in which case the pacer above meters packets out at the
rate it measures and the difference accumulates in the datagram send buffer.
Nothing is lost. Everything is late.
This was measured over a 1000-mile link: every frame arrived, every frame
completed, zero QUIC loss, zero congestion events, and a **flat** 180 ms round
trip for an entire session — while the picture ran up to eight seconds behind.
A flat RTT is the tell that the queue is *ours* and not the network's; a
standing queue in the path would have shown up as rising round-trip time.
The trap is that every health signal reads clean. Loss only appears once the
buffer finally overflows, long after the session became unplayable, and by then
the queue is deep enough that the only way out is the floor. A controller
driven by loss alone therefore climbs to its ceiling against a path it is
already overrunning, collapses when the buffer spills, drains, and climbs
again — a sawtooth with a period of tens of seconds, in which the stream is
never once at a rate the path can carry.
`DGRAM_BUFFER_BYTES` is 4 MiB. That was chosen as headroom for a keyframe
burst, which is a real requirement and still correct. But a buffer sized in
bytes is a queue whose *duration* depends on the drain rate: at a couple of
megabits, 4 MiB is several seconds of video. Sizing for one dimension silently
set a bound in the other.
The signal that does work was available the whole time and simply never read:
```
backlog_bytes = DGRAM_BUFFER_BYTES - conn.datagram_send_buffer_space()
backlog_ms = backlog_bytes * 8 / drain_kbps
```
Divide by the rate the queue actually drains at — what is getting through — and
not by what is being asked for, because the latter is the number that is too
high whenever this matters.
## What follows
These are the design rules we arrived at, and the reasoning is above rather
than in the code that implements them.
**Do not send what the receiver cannot use.** Frames that depend on a keyframe
the receiver is still waiting for are not merely wasted bandwidth; on this
transport they actively delay the keyframe.
**Do not leave a hole for something you chose not to send.** A receiver counts
a missing sequence number as loss, and that count feeds our own congestion
response. A sender that skips a frame *and* burns its sequence number makes the
controller lower the bitrate in response to the sender's own decision. Reusing
the number is safe precisely because nothing went out under it.
**Bound every suppression.** Withholding is correct only while the assumption
behind it holds. If the keyframe never comes, withholding forever converts a
recoverable freeze into a permanent black screen, so the suppression expires
and says that it did.
**Measure the queue, not just the casualties.** Loss says a path was overrun
some time ago. A standing send queue says it is being overrun right now, while
the frames are still only late. On a transport that drops the oldest datagram
rather than refusing the newest, the queue is the only signal that arrives in
time to act on — and climbing is what digs it, so climbing needs the queue to
be empty and not merely the loss to be low.
**Trust the receiver over the sender.** The sender's view of this path was
measured reporting 182 ms RTT with zero packet loss while the client received
almost nothing. Some of that gap is structural, per the section above. The far
end's report of what actually arrived is the only measurement that cannot be
quietly wrong in our favour.
## The method
Every item here was found by reading the dependency's source, and every one of
them contradicted something we believed. The first draft of this very file
cited the wrong crate — upstream quinn rather than the fork iroh actually
builds against — which is a good illustration of how little intuition is worth
here even when the conclusions survive. The same thing happened one layer
down, in Vulkan Video: rate control turned out to be session state rather than
per-frame state, `gopFrameCount` of zero means "implementation chooses" rather
than "infinite" (`UINT32_MAX` means infinite), and a hardcoded QP ceiling made
low bitrate targets silently unreachable. None of that was in any comment we
had written, and two of those we had confidently asserted the opposite of.
So: **when behaviour depends on a dependency's internals, read them, and write
down what you found with a path to check it against.** Not the documentation,
which is frequently about intent; not the interface, which is frequently about
what you may call rather than what will happen. A claim in this file that
nobody can verify in an afternoon is a claim that will be wrong within a year
and believed anyway.