mirror of
https://github.com/nestriness/nestri.git
synced 2026-09-19 17:25:19 +03:00
6164e0c636
A Vulkan implicit layer that captures frames from inside the workload's own process and encodes them on the GPU they were drawn on. Fourth and last of this batch, imported as a tree from `nestrilabs/nescapture` on the same terms. Filed under `apps/` rather than `crates/` despite building a cdylib. The rule here is what a thing *is*, not what it compiles to: this is a finished artefact that gets installed into an image beside its layer manifest, not a library another crate in this tree depends on. `crates/` is for the latter, and putting this there would make the distinction useless the first time someone looked. Wired to the workspace, `nesprotocol` by path. Its description named the transport component; that reads better as what it actually is — where the frames go — so it says that instead. Whole workspace builds and tests: 21 across four members. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
210 lines
8.4 KiB
Markdown
210 lines
8.4 KiB
Markdown
# nescapture
|
|
|
|
A Vulkan implicit layer that captures frames from a running game, encodes them
|
|
with **Vulkan Video** hardware acceleration (H.264 / H.265 / AV1), packetizes with
|
|
Reed-Solomon FEC, and streams over RTP/UDP to a Moonlight-compatible client —
|
|
all with **zero CPU copies** — fully GPU from game rendering to RTP output.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Game process
|
|
│ Vulkan calls
|
|
▼
|
|
┌───────────────────────────────────────────────┐
|
|
│ nescapture Vulkan implicit layer │
|
|
│ │
|
|
│ vkCreateShaderModule → SHA-256 hash │
|
|
│ vkCreateGraphicsPipelines → track hashes │
|
|
│ vkCmdBindPipeline → detect HUD shaders │
|
|
│ vkCmdEndRenderPass/Rendering → inject copy │
|
|
│ vkQueuePresentKHR → capture + encode │
|
|
└───────────────────────────────────────────────┘
|
|
│ GPU blit (same device)
|
|
▼
|
|
final_image (DMA-BUF exportable)
|
|
│ get_dmabuf_fd(final_memory)
|
|
▼
|
|
DmaBufImporter (pixelforge VkDevice)
|
|
│ import_or_reuse() → vk::Image
|
|
▼
|
|
ColorConverter (GPU compute shader)
|
|
│ BGRA/RGB10/FP16 → NV12/P010/YUV444
|
|
▼
|
|
Encoder (Vulkan Video, hardware H.264/H.265)
|
|
│ encode()
|
|
▼
|
|
EncodedPacket (Annex-B)
|
|
│
|
|
▼
|
|
Packetizer (Moonlight wire format, Reed-Solomon FEC)
|
|
│ UDP datagrams
|
|
▼
|
|
Moonlight client (or any RTP/UDP receiver)
|
|
|
|
CPU fallback: only when DMA-BUF export unavailable (rare driver config)
|
|
```
|
|
|
|
---
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# 1. Build
|
|
cargo build --release
|
|
|
|
# 2. Install the layer manifest
|
|
sudo cp manifest/VK_LAYER_nescapture.json /usr/share/vulkan/implicit_layer.d/
|
|
# Edit the manifest's `library_path` to point at target/release/libnescapture.so
|
|
|
|
# 3. Configure and launch a game
|
|
export NESCAPTURE_ENABLE=1
|
|
export NESCAPTURE_RTP_HOST=192.168.1.50 # Moonlight / receiver IP
|
|
export NESCAPTURE_RTP_PORT=47998 # default
|
|
export NESCAPTURE_CODEC=h265 # h264 | h265 | av1 (auto-probes if unset)
|
|
export NESCAPTURE_BITRATE=10000 # kbps (CBR; ignored if NESCAPTURE_QP is set)
|
|
export NESCAPTURE_FPS=60
|
|
export RUST_LOG=info # or NESCAPTURE_LOG=debug
|
|
|
|
wine MyGame.exe # or native Vulkan game
|
|
```
|
|
|
|
---
|
|
|
|
## Environment variables
|
|
|
|
| Variable | Default | Description |
|
|
| ------------------------- | -------------- | ------------------------------------------------------ |
|
|
| `NESCAPTURE_ENABLE` | _(unset)_ | Set to `1` to activate the layer |
|
|
| `NESCAPTURE_RTP_HOST` | _(required)_ | Destination IP / hostname for RTP stream |
|
|
| `NESCAPTURE_RTP_PORT` | `47998` | Destination UDP port |
|
|
| `NESCAPTURE_CODEC` | auto | `h264`, `h265` or `av1` — falls back to h264 if unsupported |
|
|
| `NESCAPTURE_BITRATE` | `10000` | CBR target bitrate in kbps |
|
|
| `NESCAPTURE_QP` | _(unset)_ | If set, use CQP with this quality level instead of CBR |
|
|
| `NESCAPTURE_FPS` | `60` | Target frame rate |
|
|
| `NESCAPTURE_IDR_INTERVAL` | `120` | Force an IDR keyframe every N frames |
|
|
| `NESCAPTURE_FEC_PCT` | `20` | Reed-Solomon FEC percentage |
|
|
| `NESCAPTURE_MIN_FEC` | `2` | Minimum FEC packets per block |
|
|
| `NESCAPTURE_PACKET_SIZE` | `1392` | Max UDP payload size (bytes) |
|
|
| `NESCAPTURE_CTRL_PORT` | `47999` | UDP port for the control stream |
|
|
| `NESCAPTURE_CAPTURE_HUDLESS` | _(unset)_ | Set to `1` to also capture HUDless frames |
|
|
| `NESCAPTURE_CONFIG` | _(unset)_ | Path to per-game shader-hash TOML config |
|
|
| `NESCAPTURE_GAME_NAME` | (exe basename) | Override game identification |
|
|
| `NESCAPTURE_DISCOVER` | _(unset)_ | Set to `1` to enable discovery mode (logs all draws) |
|
|
| `NESCAPTURE_LOG` | `info` | Log level (`error`, `warn`, `info`, `debug`, `trace`) |
|
|
| `NESCAPTURE_RTP_FORMAT` | `moonlight` | `standard` for RFC 6184/7798 RTP (GStreamer/FFmpeg compatible), `moonlight` for Moonlight wire format with FEC |
|
|
|
|
---
|
|
|
|
## Control stream
|
|
|
|
Send single-byte UDP datagrams to `NESCAPTURE_CTRL_PORT` (default 47999):
|
|
|
|
| Byte | Command |
|
|
| ------ | -------------------- |
|
|
| `0x01` | Start streaming |
|
|
| `0x02` | Stop streaming |
|
|
| `0x03` | Request IDR keyframe |
|
|
|
|
Example with netcat:
|
|
|
|
```bash
|
|
# Request IDR
|
|
printf '\x03' | nc -u -q1 localhost 47999
|
|
```
|
|
|
|
The control handle wiring in `present.rs` is currently a commented-out stub.
|
|
See `control.rs` for the full wiring instructions.
|
|
|
|
---
|
|
|
|
## Per-game shader-hash config
|
|
|
|
HUD shader detection requires a per-game config. Run the game once with
|
|
`NESCAPTURE_DISCOVER=1` to log all shaders, then identify HUD pipelines by the
|
|
`[SUSPECT]` marker (blend=true, depth=false, ≤6 vertices).
|
|
|
|
```toml
|
|
# ~/.config/nescapture/games.toml
|
|
[game."GameName.exe"]
|
|
hud_fragment_shaders = ["0xaabbccddeeff0011"]
|
|
hud_vertex_shaders = ["0x1a2b3c4d5e6f7890"]
|
|
skip_fragment_shaders = ["0x1122334455667788"]
|
|
```
|
|
|
|
```bash
|
|
export NESCAPTURE_CONFIG=~/.config/nescapture/games.toml
|
|
export NESCAPTURE_GAME_NAME=GameName.exe
|
|
```
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
| Crate | Purpose |
|
|
| ---------------------- | -------------------------------------------- |
|
|
| `pixelforge` | Vulkan Video hardware encode (H.264 / H.265) |
|
|
| `ash` | Vulkan bindings |
|
|
| `reed-solomon-erasure` | FEC for RTP packetizer |
|
|
| `sha2` + `bytemuck` | SPIR-V shader fingerprinting |
|
|
| `dashmap` | Lock-free concurrent state maps |
|
|
| `serde` + `toml` | Per-game shader config |
|
|
|
|
---
|
|
|
|
## Zero-copy GPU pipeline
|
|
|
|
The full GPU path is implemented — no CPU color conversion:
|
|
|
|
1. `final_image` allocated with `DMA_BUF_EXT` external memory (`capture.rs`).
|
|
2. `get_dmabuf_fd()` exports the DMA-BUF fd (`capture.rs`).
|
|
3. `DmaBufImporter::import_or_reuse()` imports the fd as a `vk::Image` (`dmabuf_import.rs`).
|
|
4. `ColorConverter::convert()` runs a GPU compute shader:
|
|
BGRA/RGBA/RGB10/FP16 → NV12/P010/YUV444 (`encode.rs`).
|
|
5. `Encoder::encode()` encodes from the converter's output directly.
|
|
6. RTP packetizer + UDP send.
|
|
|
|
CPU fallback exists only for rare driver configurations that don't support
|
|
DMA-BUF external memory export.
|
|
|
|
---
|
|
|
|
## File structure
|
|
|
|
```
|
|
nescapture/
|
|
├── Cargo.toml
|
|
├── README.md
|
|
├── ARCHITECTURE.md
|
|
├── manifest/
|
|
│ └── VK_LAYER_nescapture.json
|
|
└── src/
|
|
├── lib.rs — entry points, dispatch routing
|
|
├── dispatch.rs — Vulkan function-pointer types + tables
|
|
├── state.rs — global DashMaps, DeviceState, CbState
|
|
├── instance.rs — vkCreateInstance / vkDestroyInstance
|
|
├── device.rs — vkCreateDevice / vkDestroyDevice
|
|
├── shader.rs — SPIR-V hashing
|
|
├── pipeline.rs — graphics pipeline tracking
|
|
├── framebuffer.rs — image view / framebuffer tracking
|
|
├── commands.rs — vkCmdBind*, vkCmdDraw*, vkCmdBeginRenderPass
|
|
├── swapchain.rs — vkCreateSwapchainKHR, image enumeration
|
|
├── capture.rs — GPU blit to capture image, DMA-BUF export
|
|
├── present.rs — vkQueuePresentKHR, encode dispatch
|
|
├── encode.rs — pixelforge pipeline, codec probing, RTP send
|
|
├── dmabuf_import.rs — DmaBufImporter (cross-device zero-copy import)
|
|
├── packetizer.rs — Moonlight RTP packetizer (Reed-Solomon FEC)
|
|
├── shard_batch.rs — zero-alloc shard buffer
|
|
├── control.rs — UDP control stream (IDR / start / stop)
|
|
├── config.rs — per-game TOML shader-hash config
|
|
└── discovery.rs — draw-call logging for shader discovery
|
|
```
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
TBD
|