mirror of
https://github.com/nestriness/nestri.git
synced 2026-09-19 17:25:19 +03:00
6ea241c910
All three came across with their standalone repos and none had been touched since the transport changed. nescapture claimed to packetize with Reed-Solomon FEC and stream RTP/UDP to a Moonlight client. It sends encoded frames to neshub over a Unix socket. packetizer.rs, control.rs and shard_batch.rs do not exist, and neither does ARCHITECTURE.md. Nine documented environment variables are not read by anything -- NESCAPTURE_RTP_HOST was listed as *required* -- and five that are read were undocumented, including the one that says where frames go. Someone following that quick start would have set a required variable that does nothing and got no output. Documented the three sockets, since nescapture binds one and connects to two and that was written down nowhere. neswire documented --rtp-addr and --channels against a gstreamer pipeline. It has --ipc-path, --channels, --packet-duration-ms and --bitrate-per-channel, and hub-stub exists precisely so it can be tested without a hub. Kept the reason hub-stub decodes rather than counts bytes: Opus codes silence at ~3 kbps, so a dead sink looks alive on a meter. nescope was mostly right. It called the capture layer "vkcapture", listed seven of fifteen modules and five of nine flags, and its TODO list was three items that neshub and nescapture now do. Added compositor mode, which is the shape a real session uses and was undocumented. All three licence lines were "TBD" or "See project repository".
177 lines
6.1 KiB
Markdown
177 lines
6.1 KiB
Markdown
# nescapture
|
|
|
|
A Vulkan implicit layer that captures frames from inside the workload's own
|
|
process, encodes them with **Vulkan Video** on the GPU that drew them, and
|
|
sends them to [`neshub`](../neshub) over a Unix socket — no copy out to the
|
|
CPU and back.
|
|
|
|
Being a layer rather than a screen-scraper is the whole point: the frame is
|
|
already on the GPU when we get it, and it never leaves.
|
|
|
|
---
|
|
|
|
## Where it sits
|
|
|
|
```
|
|
Game process
|
|
│ Vulkan calls
|
|
▼
|
|
┌──────────────────────────────────────────────┐
|
|
│ nescapture implicit layer │
|
|
│ │
|
|
│ vkCreateShaderModule → SHA-256 hash │
|
|
│ vkCreateGraphicsPipelines → track hashes │
|
|
│ vkCmdBindPipeline → detect HUD │
|
|
│ vkQueuePresentKHR → capture+encode │
|
|
└──────────────────────────────────────────────┘
|
|
│ GPU blit, same device
|
|
▼
|
|
final_image (DMA-BUF exportable)
|
|
│ get_dmabuf_fd(final_memory)
|
|
▼
|
|
DmaBufImporter (pixelforge VkDevice)
|
|
│ import_or_reuse() → vk::Image
|
|
▼
|
|
ColorConverter (GPU compute shader)
|
|
│ BGRA/RGB10/FP16 → NV12/P010/YUV444
|
|
▼
|
|
Encoder (Vulkan Video: H.264 / H.265 / AV1)
|
|
│ Annex-B packets
|
|
▼
|
|
Unix datagram → neshub → the client
|
|
```
|
|
|
|
CPU fallback exists only for driver configurations without DMA-BUF external
|
|
memory export.
|
|
|
|
---
|
|
|
|
## Sockets
|
|
|
|
| path | direction | carries |
|
|
| --- | --- | --- |
|
|
| `/tmp/nestri-video.sock` | nescapture → neshub | encoded frames |
|
|
| `/tmp/nestri-stats.sock` | nescapture → neshub | capture fps, encode ms, drops |
|
|
| `/tmp/nescapture-cmd.sock` | neshub → nescapture | IDR requests, encode settings |
|
|
|
|
nescapture binds the command socket and connects to the other two. The stats
|
|
path is derived from `NESCAPTURE_IPC_PATH`'s directory, so moving the video
|
|
socket moves both.
|
|
|
|
---
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# 1. Build
|
|
cargo build --release -p nescapture
|
|
|
|
# 2. Install the layer manifest
|
|
sudo cp apps/nescapture/manifest/VK_LAYER_nescapture.json /usr/share/vulkan/implicit_layer.d/
|
|
# Point the manifest's `library_path` at target/release/libnescapture_layer.so
|
|
|
|
# 3. Run something that draws
|
|
export NESCAPTURE_ENABLE=1
|
|
export NESCAPTURE_CODEC=h265
|
|
export NESCAPTURE_BITRATE=10000
|
|
export RUST_LOG=info
|
|
|
|
./my-vulkan-app
|
|
```
|
|
|
|
The layer is inert unless `NESCAPTURE_ENABLE=1`. That is deliberate — an
|
|
implicit layer is loaded into *every* Vulkan process on the system.
|
|
|
|
---
|
|
|
|
## Environment variables
|
|
|
|
| Variable | Default | Description |
|
|
| --- | --- | --- |
|
|
| `NESCAPTURE_ENABLE` | _(unset)_ | Set to `1` to activate the layer. Nothing happens otherwise |
|
|
| `NESCAPTURE_IPC_PATH` | `/tmp/nestri-video.sock` | Where to send encoded frames |
|
|
| `NESCAPTURE_CODEC` | best available | `h264`, `h265` or `av1`; probes if unset |
|
|
| `NESCAPTURE_FORMAT` | `yuv420` | `yuv420` or `yuv444` |
|
|
| `NESCAPTURE_DEPTH` | auto | `8` or `10`; inferred from the swapchain `VkFormat` if unset |
|
|
| `NESCAPTURE_BITRATE` | `10000` | CBR target in kbps. Ignored when `NESCAPTURE_QP` is set |
|
|
| `NESCAPTURE_QP` | _(unset)_ | Constant QP instead of CBR |
|
|
| `NESCAPTURE_FPS` | `60` | Target frame rate |
|
|
| `NESCAPTURE_IDR_INTERVAL` | `4` | Force an IDR every N **seconds** |
|
|
| `NESCAPTURE_TUNE` | _(unset)_ | `highquality`, `lowlatency`, `ultralowlatency`, `lossless` |
|
|
| `NESCAPTURE_CONFIG` | _(unset)_ | Path to the per-app shader-hash TOML |
|
|
| `NESCAPTURE_GAME_NAME` | exe basename | Override app identification for that config |
|
|
| `NESCAPTURE_DISCOVER` | _(unset)_ | Set to `1` to log every draw, for finding HUD shaders |
|
|
| `RUST_LOG` | `error` | Standard `env_logger` filter, e.g. `nescapture_layer=debug` |
|
|
|
|
Everything else is decided at runtime: the client asks `neshub` for a codec or
|
|
bitrate change and it arrives on the command socket, so the encoder is
|
|
reconfigured without a restart.
|
|
|
|
---
|
|
|
|
## Per-app shader-hash config
|
|
|
|
HUD detection needs to know which pipelines draw the HUD, and that is
|
|
per-application. Run once with `NESCAPTURE_DISCOVER=1` to log every shader,
|
|
then pick out the HUD pipelines by the `[SUSPECT]` marker — blend on, depth
|
|
off, six vertices or fewer.
|
|
|
|
```toml
|
|
# ~/.config/nescapture/apps.toml
|
|
[game."MyApp.exe"]
|
|
hud_fragment_shaders = ["0xaabbccddeeff0011"]
|
|
hud_vertex_shaders = ["0x1a2b3c4d5e6f7890"]
|
|
skip_fragment_shaders = ["0x1122334455667788"]
|
|
```
|
|
|
|
```bash
|
|
export NESCAPTURE_CONFIG=~/.config/nescapture/apps.toml
|
|
export NESCAPTURE_GAME_NAME=MyApp.exe
|
|
```
|
|
|
|
---
|
|
|
|
## Modules
|
|
|
|
```
|
|
src/
|
|
├── lib.rs entry points, dispatch routing
|
|
├── dispatch.rs Vulkan function-pointer types and tables
|
|
├── state.rs global DashMaps, DeviceState, CbState
|
|
├── instance.rs vkCreateInstance / vkDestroyInstance
|
|
├── device.rs vkCreateDevice / vkDestroyDevice
|
|
├── shader.rs SPIR-V hashing
|
|
├── pipeline.rs graphics pipeline tracking
|
|
├── framebuffer.rs image view and framebuffer tracking
|
|
├── commands.rs vkCmdBind*, vkCmdDraw*, vkCmdBeginRenderPass
|
|
├── swapchain.rs vkCreateSwapchainKHR, image enumeration
|
|
├── capture.rs GPU blit to the capture image, DMA-BUF export
|
|
├── present.rs vkQueuePresentKHR, encode dispatch
|
|
├── encode.rs pixelforge pipeline, codec probing, IPC send
|
|
├── dmabuf_import.rs cross-device zero-copy import
|
|
├── config.rs per-app TOML shader-hash config
|
|
└── discovery.rs draw-call logging for shader discovery
|
|
```
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
| Crate | Purpose |
|
|
| --- | --- |
|
|
| `pixelforge` | Vulkan Video hardware encode |
|
|
| `ash` | Vulkan bindings |
|
|
| `nesprotocol` | The IPC frame format `neshub` reads |
|
|
| `sha2`, `bytemuck` | SPIR-V fingerprinting |
|
|
| `dashmap`, `once_cell` | Lock-free concurrent state |
|
|
| `serde`, `toml` | Per-app shader config |
|
|
|
|
`ash` and `pixelforge` are pinned to git revisions — both track Vulkan Video
|
|
support that has not landed in a release.
|
|
|
|
---
|
|
|
|
## Licence
|
|
|
|
Apache 2.0. See [LICENSE](../../LICENSE).
|