github-mirrors/netris-nestri
1
0
Fork 0
mirror of https://github.com/nestriness/nestri.git synced 2026-09-27 04:52:25 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: media bitrate control, HDR (#346)

Browse Source 
Fixes: #335 

Still a work-in-progress.

---------

Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Wanjohi <elviswanjohi47@gmail.com>
This commit is contained in:
Kristian Ollikainen
2026-09-25 12:13:34 +03:00
committed by GitHub
co-authored by DatCaptainHorse Claude Opus 5 Wanjohi
parent 1c721962f4
commit 0811f57f1a
64 changed files with 15151 additions and 2702 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/nescapture/Cargo.toml
+2 -2
View File
@@ -32,9 +32,9 @@ toml = "0.8"
serde = { version = "1", features = ["derive"] }
# Vulkan Video hardware encoding.
pixelforge = { git = "https://github.com/DatCaptainHorse/pixelforge.git", rev = "681fa4dd8bce5dabf008d00983e991b0eb8b3696", features = ["dmabuf"] }
pixelforge = { git = "https://github.com/DatCaptainHorse/pixelforge.git", rev = "b4b7b36af6f9cbd0fedae220915edd75de933743" }
# libc for DMA-BUF OS primitives
# libc for socket options
libc = "0.2"
# Shared IPC protocol
apps/nescapture/README.md
+84 -13
View File
@@ -24,15 +24,12 @@ Game process
│ vkCmdBindPipeline → detect HUD │
│ vkQueuePresentKHR → capture+encode │
└──────────────────────────────────────────────┘
│ GPU blit, same device
│ GPU blit on the game's queue, signals a timeline semaphore
▼
final_image (DMA-BUF exportable)
│ get_dmabuf_fd(final_memory)
ring slot (an image on the game's own device)
│ read in place, once the blit's point is reached
▼
DmaBufImporter (pixelforge VkDevice)
│ import_or_reuse() → vk::Image
▼
ColorConverter (GPU compute shader)
ColorConverter (GPU compute shader, the encoder's own queue)
│ BGRA/RGB10/FP16 → NV12/P010/YUV444
▼
Encoder (Vulkan Video: H.264 / H.265 / AV1)
@@ -41,8 +38,24 @@ Encoder (Vulkan Video: H.264 / H.265 / AV1)
Unix datagram → neshub → the client
```
CPU fallback exists only for driver configurations without DMA-BUF external
memory export.
The encoder runs on the game's own `VkDevice`. The layer creates that device
with what the encoder needs: the extensions and feature bits it asks for, and
queues of its own wherever a queue family has one to spare, since a `VkQueue`
may not be submitted to from two threads at once. Where no family has room,
the game's queue is created internally synchronized and shared. Every step of
a frame is ordered on the GPU; nothing waits on the CPU.
Where the conversion is only the YUV matrix and the device has
`VK_VALVE_video_encode_rgb_conversion`, the encoder takes the RGB frame and
converts it itself, and the converter is not built. That path runs at limited
range, since the one driver offering it writes limited range whatever it is
asked; everything else is full range.
When the game's device cannot host the encoder -- an instance the loader
cannot raise to Vulkan 1.1, no queue the encoder could safely use, a driver
refusing the additions, or `NESCAPTURE_SHARED_DEVICE=0` -- the encoder gets a
device of its own, and each frame is read back on the CPU and uploaded there.
That works everywhere and costs a copy each way.
---
@@ -93,11 +106,18 @@ implicit layer is loaded into *every* Vulkan process on the system.
| `NESCAPTURE_CODEC` | best available | `h264`, `h265` or `av1`; probes if unset |
| `NESCAPTURE_FORMAT` | `yuv420` | `yuv420` or `yuv444` |
| `NESCAPTURE_DEPTH` | auto | `8` or `10`; inferred from the swapchain `VkFormat` if unset |
| `NESCAPTURE_BITRATE` | `10000` | CBR target in kbps. Ignored when `NESCAPTURE_QP` is set |
| `NESCAPTURE_QP` | _(unset)_ | Constant QP instead of CBR |
| `NESCAPTURE_RC` | _(inferred)_ | `cqp`, `cbr` or `vbr`. Unset infers `cqp` when `NESCAPTURE_QP` is set, `cbr` otherwise |
| `NESCAPTURE_BITRATE` | `10000` | Target bitrate in kbps, under `cbr` and `vbr` |
| `NESCAPTURE_BITRATE_MAX` | 1.5x the target | VBR ceiling in kbps. Ignored outside `vbr` |
| `NESCAPTURE_QP` | _(unset)_ | Constant QP, under `cqp` |
| `NESCAPTURE_FPS` | `60` | Target frame rate |
| `NESCAPTURE_IDR_INTERVAL` | `4` | Force an IDR every N **seconds** |
| `NESCAPTURE_INTRA_REFRESH` | _(off)_ | Set to `1` to replace periodic key frames with an intra refresh cycle |
| `NESCAPTURE_INTRA_REFRESH_QP_DELTA` | `-4` | QP shift inside the refresh band; negative spends bits on it |
| `NESCAPTURE_INTRA_REFRESH_SHAPE` | auto | `rows`, `columns` or `partitions`; the driver chooses if unset |
| `NESCAPTURE_TUNE` | _(unset)_ | `highquality`, `lowlatency`, `ultralowlatency`, `lossless` |
| `NESCAPTURE_SHARED_DEVICE` | _(on)_ | Set to `0` to leave the game's device as the game asked for it, and encode on a device of the encoder's own with CPU readback |
| `NESCAPTURE_RGB_ENCODE` | _(on)_ | Set to `0` to always convert with the shader, even where the encoder could convert RGB itself |
| `NESCAPTURE_CONFIG` | _(unset)_ | Path to the per-app shader-hash TOML |
| `NESCAPTURE_GAME_NAME` | exe basename | Override app identification for that config |
| `NESCAPTURE_DISCOVER` | _(unset)_ | Set to `1` to log every draw, for finding HUD shaders |
@@ -107,6 +127,57 @@ Everything else is decided at runtime: the client asks `neshub` for a codec or
bitrate change and it arrives on the command socket, so the encoder is
reconfigured without a restart.
### Intra refresh
Instead of a key frame every few seconds, each picture codes one slice of the
image as intra, so after a full cycle every part has been refreshed. The same
cost, paid evenly, with no picture much larger than any other — which is what
a link with a latency budget wants, since a key frame is the largest frame
there is.
**The cycle length is not configurable, deliberately.** It is bounded by how
many refresh regions the picture actually has, and that depends on the codec's
block size: at 1080p an H.265 picture is 17 CTB rows tall where an H.264 one is
68 macroblock rows, so the same duration is comfortable for one codec and
impossible for the other. The encoder knows the codec, the resolution and what
the device allows, and derives it from the key frame interval it replaces.
The refresh here spreads cost only; it does not make the cycle a recovery
point. Doing that would restrict prediction on every picture — expensive, and
what turns the refreshed band into a visible discontinuity — to buy a
guarantee this stream gets more cheaply from the client asking for an IDR.
The band is coded intra every cycle, so it carries none of the refinement its
neighbours have accumulated and reads as a strip of lower quality sweeping
across the picture. `NESCAPTURE_INTRA_REFRESH_QP_DELTA` spends bits back into
it, out of the rest of the frame. Measured at 1080p with ColorVideoVDP, `-4`
recovers a fifth of what intra refresh costs and the encoded size does not
grow — but the best value depends on the content, and too large a shift
starves the rest of the frame faster than too small a one helps. Devices that
cannot express a negative delta, or whose refresh regions follow the slice
layout rather than a block sweep, decline it and say so.
`NESCAPTURE_INTRA_REFRESH_SHAPE` stays configurable because the device cannot
answer it: whether a horizontal or vertical sweep looks better depends on how
the content moves. It also changes how many regions there are — a 1080p
picture in 64×64 blocks is 17 rows but 30 columns, so `columns` allows a
longer cycle and thus less intra per picture.
### Rate control
`cbr` holds every frame to the same size, which is what a link with a fixed
budget wants. `vbr` holds the same *average* while letting a frame that needs
it spend up to the ceiling — a scene change is coded rather than smeared, at
the cost of a burst the path has to absorb. `cqp` holds quality constant and
lets the bitrate go wherever the content takes it, which is a recording
setting rather than a streaming one.
The command socket carries a target bitrate but has no way to name a mode
beyond CBR and constant QP. A target arriving while the encode is `vbr` is
therefore applied as a target, leaving the mode and the ceiling alone — so the
ceiling asked for at launch survives a session, and a congestion controller
adjusts underneath it. Retargeting costs no rebuild and no key frame.
---
## Per-app shader-hash config
@@ -145,10 +216,10 @@ src/
├── framebuffer.rs image view and framebuffer tracking
├── commands.rs vkCmdBind*, vkCmdDraw*, vkCmdBeginRenderPass
├── swapchain.rs vkCreateSwapchainKHR, image enumeration
├── capture.rs GPU blit to the capture image, DMA-BUF export
├── capture.rs GPU blit into the capture ring, CPU readback fallback
├── present.rs vkQueuePresentKHR, encode dispatch
├── encode.rs pixelforge pipeline, codec probing, IPC send
├── dmabuf_import.rs cross-device zero-copy import
├── shared.rs creating the game's device for the encoder to share
├── config.rs per-app TOML shader-hash config
└── discovery.rs draw-call logging for shader discovery
```
apps/nescapture/scripts/verify-chain.sh
+3 -2
View File
@@ -4,7 +4,7 @@
# Runs a Vulkan workload under the compositor with the layer active, then checks
# the encoded result against the compositor's own readback of the same frames.
# Two independent paths see the same content: the compositor reads the surface
# back to the CPU, the layer exports it as a DMA-BUF and encodes it on the GPU.
# back to the CPU, the layer encodes it on the GPU without it leaving the device.
# Agreement between them is the evidence; a single path cannot tell a correct
# frame from a plausible-looking wrong one.
#
@@ -96,7 +96,8 @@ wait $RECV || true
FRAMES="$(cat "$WORK/frames.txt")"
echo
echo "frames encoded: $FRAMES"
grep -m1 "First import" "$WORK/run.log" || echo " (no DMA-BUF import logged)"
grep -m1 "encoding on the game's own device" "$WORK/run.log" \
|| echo " (encoding on a device of its own, with CPU readback)"
python3 - "$WORK" "$STREAM" "$FRAMES" <<'PY'
import glob, subprocess, sys
apps/nescapture/src/capture.rs
+818 -287
View File
File diff suppressed because it is too large Load Diff
apps/nescapture/src/device.rs
+163 -86
View File
@@ -14,6 +14,7 @@ use std::os::raw::c_void;
use std::sync::Arc;
const VK_LAYER_LINK_INFO: u32 = 0;
const VK_LOADER_DATA_CALLBACK: u32 = 1;
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkCreateDevice(
@@ -43,6 +44,14 @@ pub unsafe extern "system" fn vkCreateDevice(
};
unsafe { (*layer_info).u.pDeviceLayerInfo = (*dev_link).pNext };
let set_loader_data = unsafe {
find_layer_link::<VkLayerDeviceCreateInfo>(
(*p_create_info).p_next as *const c_void,
VK_LOADER_DATA_CALLBACK,
)
}
.and_then(|info| unsafe { (*info).u.pfnSetDeviceLoaderData });
let inst_key = unsafe { dispatch_key(physical_device.as_raw() as *const c_void) };
let istate = match INSTANCE_STATE.get(&inst_key) {
Some(s) => s.clone(),
@@ -54,11 +63,8 @@ pub unsafe extern "system" fn vkCreateDevice(
}
};
// ── Inject DMA-BUF extensions for zero-copy capture ──────────────────
let ci = unsafe { &*p_create_info };
// Collect the game's original extensions.
let original_extensions: Vec<*const libc::c_char> =
let game_extensions: Vec<*const libc::c_char> =
if ci.enabled_extension_count > 0 && !ci.pp_enabled_extension_names.is_null() {
unsafe {
std::slice::from_raw_parts(
@@ -71,69 +77,34 @@ pub unsafe extern "system" fn vkCreateDevice(
Vec::new()
};
// Extensions we need — static byte strings so pointers stay valid.
const EXT_EXTERNAL_MEMORY: &[u8] = b"VK_KHR_external_memory\0";
const EXT_EXTERNAL_MEMORY_FD: &[u8] = b"VK_KHR_external_memory_fd\0";
const EXT_EXTERNAL_MEMORY_DMABUF: &[u8] = b"VK_EXT_external_memory_dma_buf\0";
// Lets the capture ring be allocated tiled. The importer has always created
// its side with DRM_FORMAT_MODIFIER_EXT tiling; without this the producer
// can only offer it a linear buffer.
const EXT_IMAGE_DRM_FORMAT_MODIFIER: &[u8] = b"VK_EXT_image_drm_format_modifier\0";
let needed: &[&[u8]] = &[
EXT_EXTERNAL_MEMORY,
EXT_EXTERNAL_MEMORY_FD,
EXT_EXTERNAL_MEMORY_DMABUF,
EXT_IMAGE_DRM_FORMAT_MODIFIER,
];
// Build extended list: original + any of ours not already present.
let mut extended = original_extensions.clone();
for &ext in needed {
let name_cstr = unsafe { std::ffi::CStr::from_bytes_with_nul_unchecked(ext) };
let already = extended
.iter()
.any(|&ptr| unsafe { std::ffi::CStr::from_ptr(ptr) == name_cstr });
if !already {
extended.push(ext.as_ptr() as *const libc::c_char);
// Where it can, the encoder runs on this very device, so the device is
// first created with what that needs. Anything refused falls back to the
// device exactly as the game asked for it, and the encoder to a device of
// its own.
let mut shared = None;
if let Some(prepared) =
unsafe { crate::shared::prepare(&istate, next_gdpa, physical_device, ci, &game_extensions) }
{
let shared_ci = prepared.create_info(ci);
let result =
unsafe { (istate.create_device)(physical_device, &shared_ci, p_allocator, p_device) };
let (additions, entry, instance) = unsafe { prepared.finish() };
if result == vk::Result::SUCCESS {
shared = Some((additions, entry, instance));
} else {
log::warn!(
"vkCreateDevice with the encoder's additions failed ({result:?}), \
retrying as the game asked"
);
}
}
// Try with injected extensions first.
//
// The device's queue create info is passed through unchanged. An earlier
// version bumped the first family's queue count by one to get a dedicated
// capture queue, which was then never used — and could not be: the capture
// blit has to be submitted to the queue the game presents on, or it gains
// no ordering against the present. All the bump did was risk exceeding the
// family's available queue count on the way in.
let mut modified_ci = *ci;
modified_ci.enabled_extension_count = extended.len() as u32;
modified_ci.pp_enabled_extension_names = extended.as_ptr();
let mut dmabuf_available = true;
let result =
unsafe { (istate.create_device)(physical_device, &modified_ci, p_allocator, p_device) };
let result = if result != vk::Result::SUCCESS {
// Driver rejected our extensions — retry with original create info.
log::warn!(
"vkCreateDevice with DMA-BUF extensions failed ({:?}), \
retrying without — CPU readback fallback will be used",
result
);
dmabuf_available = false;
unsafe { (istate.create_device)(physical_device, p_create_info, p_allocator, p_device) }
} else {
log::info!("DMA-BUF extensions injected successfully");
result
};
if result != vk::Result::SUCCESS {
return result;
}
if !dmabuf_available {
log::warn!("DMA-BUF extensions missing — will use CPU readback fallback (expensive!)");
if shared.is_none() {
let result = unsafe {
(istate.create_device)(physical_device, p_create_info, p_allocator, p_device)
};
if result != vk::Result::SUCCESS {
return result;
}
}
let device = unsafe { *p_device };
@@ -156,6 +127,7 @@ pub unsafe extern "system" fn vkCreateDevice(
get_device_proc_addr: next_gdpa,
destroy_device: load!(b"vkDestroyDevice\0"),
get_device_queue: load!(b"vkGetDeviceQueue\0"),
get_device_queue2: try_load!(b"vkGetDeviceQueue2\0"),
queue_present_khr: try_load!(b"vkQueuePresentKHR\0"),
// Phase 1
@@ -189,10 +161,6 @@ pub unsafe extern "system" fn vkCreateDevice(
cmd_pipeline_barrier: load!(b"vkCmdPipelineBarrier\0"),
cmd_copy_image: load!(b"vkCmdCopyImage\0"),
get_image_subresource_layout: load!(b"vkGetImageSubresourceLayout\0"),
get_memory_fd_khr: try_load!(b"vkGetMemoryFdKHR\0"),
get_image_drm_format_modifier_properties_ext: try_load!(
b"vkGetImageDrmFormatModifierPropertiesEXT\0"
),
create_query_pool: try_load!(b"vkCreateQueryPool\0"),
destroy_query_pool: try_load!(b"vkDestroyQueryPool\0"),
cmd_reset_query_pool: try_load!(b"vkCmdResetQueryPool\0"),
@@ -232,6 +200,17 @@ pub unsafe extern "system" fn vkCreateDevice(
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
let shared = shared.map(|(additions, entry, instance)| unsafe {
crate::shared::SharedDevice::adopt(
additions,
entry,
instance,
physical_device,
device,
next_gdpa,
)
});
// Phase 3: load shader hash config
let shader_hashes = config::resolve_config_path()
.as_ref()
@@ -252,6 +231,9 @@ pub unsafe extern "system" fn vkCreateDevice(
raw: device,
physical_device,
fp,
shared,
shared_active: std::sync::atomic::AtomicBool::new(false),
set_loader_data,
shader_registry: DashMap::new(),
pipeline_registry: DashMap::new(),
@@ -279,6 +261,7 @@ pub unsafe extern "system" fn vkCreateDevice(
}),
swapchain_colorspace: std::sync::atomic::AtomicU32::new(0),
frame_counter: std::sync::atomic::AtomicU64::new(0),
ring_generation: std::sync::atomic::AtomicU64::new(0),
hud_detected_frame: std::sync::atomic::AtomicBool::new(false),
pending_capture_frame: std::sync::atomic::AtomicBool::new(false),
@@ -287,7 +270,6 @@ pub unsafe extern "system" fn vkCreateDevice(
encoder: std::sync::Mutex::new(None),
frame_gate: std::sync::Mutex::new(crate::pacing::FrameGate::from_env()),
frame_pacer: std::sync::Mutex::new(crate::pacing::FramePacer::from_env()),
last_present_return: std::sync::Mutex::new(None),
@@ -317,23 +299,20 @@ pub unsafe extern "system" fn vkDestroyDevice(
ds.pipeline_registry.len(),
);
// ── 1. Shut down encoder pipeline (unblocks encoder + RTP threads) ───
{
let mut enc_guard = ds.encoder.lock().unwrap();
if let Some(handle) = enc_guard.take() {
handle.shutdown();
// `handle` is dropped here → drops `frame_tx` → encoder thread's
// recv_timeout returns Disconnected → encoder thread drops
// `encoded_tx` → RTP thread exits too.
//
// Give threads a moment to drain. In production you'd join the
// JoinHandles, but since we don't store them, a short sleep +
// the AtomicBool shutdown flag is sufficient.
log::info!("encoder pipeline shutdown signaled");
// ── 1. Shut down the encode pipeline ──────────────────────────────────
//
// Waited for, not just signalled. On a shared device the encoder, its
// converter and their images are objects on this very device, and
// destroying the device under them is a use-after-free. The receive
// timeout bounds how long the thread takes to notice.
let handle = ds.encoder.lock().ok().and_then(|mut g| g.take());
if let Some(handle) = handle {
if handle.finish(std::time::Duration::from_secs(2)) {
log::info!("encoder pipeline stopped");
} else {
log::error!("encoder thread did not stop in time; destroying the device anyway");
}
}
// Brief yield to let threads notice the disconnect.
std::thread::sleep(std::time::Duration::from_millis(50));
// ── 2. Tear down the capture ring ─────────────────────────────────────
//
@@ -392,13 +371,111 @@ pub unsafe extern "system" fn vkGetDeviceQueue(
) {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
if let Some(ds) = DEVICE_STATE.get(&key) {
unsafe { (ds.fp.get_device_queue)(device, queue_family_index, queue_index, p_queue) };
// A queue created internally synchronized, so the encoder can share
// it, is invisible to vkGetDeviceQueue: only vkGetDeviceQueue2 with the
// matching flags returns it. The game asked for a plain queue and gets
// this one.
match (
shares_family(&ds, queue_family_index),
ds.fp.get_device_queue2,
) {
(true, Some(get2)) => {
let info = vk::DeviceQueueInfo2::default()
.flags(vk::DeviceQueueCreateFlags::INTERNALLY_SYNCHRONIZED_KHR)
.queue_family_index(queue_family_index)
.queue_index(queue_index);
unsafe { get2(device, &info, p_queue) };
}
_ => unsafe {
(ds.fp.get_device_queue)(device, queue_family_index, queue_index, p_queue)
},
}
let queue = unsafe { *p_queue };
// The encoder's queues reach this hook without the loader in between.
// For the game's, the loader stamps them again on the way out.
unsafe { stamp(&ds, queue.as_raw() as *mut c_void) };
QUEUE_TO_DEVICE_KEY.insert(queue.as_raw(), key);
crate::state::QUEUE_TO_FAMILY.insert(queue.as_raw(), queue_family_index);
}
}
/// Give a dispatchable object this layer obtained itself the loader's
/// dispatch data.
///
/// The loader writes its dispatch pointer into every queue and command buffer
/// that passes through its own entry points. One a layer allocates by calling
/// the next layer directly never does, and a layer below this one that finds
/// its per-object state by that pointer then finds nothing: the validation
/// layer aborts, in the first vkCmd* recorded into such a command buffer. The
/// loader hands every layer this callback at vkCreateDevice for exactly this.
///
/// # Safety
///
/// `object` must be a queue or command buffer of `ds`'s device.
pub unsafe fn stamp(ds: &DeviceState, object: *mut c_void) {
if let Some(set) = ds.set_loader_data
&& unsafe { set(ds.raw, object) } != vk::Result::SUCCESS
{
log::warn!("vkSetDeviceLoaderData refused an object of the layer's own");
}
}
/// vkAllocateCommandBuffers for the encoder, whose command buffers are the
/// layer's own: allocated below the loader, so stamped here. See [`stamp`].
pub unsafe extern "system" fn encoder_allocate_command_buffers(
device: vk::Device,
p_allocate_info: *const vk::CommandBufferAllocateInfo<'_>,
p_command_buffers: *mut vk::CommandBuffer,
) -> vk::Result {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
let Some(ds) = DEVICE_STATE.get(&key).map(|s| s.clone()) else {
return vk::Result::ERROR_INITIALIZATION_FAILED;
};
let result =
unsafe { (ds.fp.allocate_command_buffers)(device, p_allocate_info, p_command_buffers) };
if result == vk::Result::SUCCESS {
let count = unsafe { (*p_allocate_info).command_buffer_count } as usize;
for cb in unsafe { std::slice::from_raw_parts(p_command_buffers, count) } {
unsafe { stamp(&ds, cb.as_raw() as *mut c_void) };
}
}
result
}
/// Whether the game's queues in `family` were created internally synchronized
/// for the encoder to share.
fn shares_family(ds: &DeviceState, family: u32) -> bool {
ds.shared
.as_ref()
.is_some_and(|s| s.queues.internally_synchronized.contains(&family))
}
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkGetDeviceQueue2(
device: vk::Device,
p_queue_info: *const vk::DeviceQueueInfo2,
p_queue: *mut vk::Queue,
) {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
let Some(ds) = DEVICE_STATE.get(&key) else {
return;
};
let Some(get2) = ds.fp.get_device_queue2 else {
return;
};
// The flags have to match the ones the queue was created with, and for a
// shared family the layer added one the game does not know about.
let mut info = unsafe { *p_queue_info };
if shares_family(&ds, info.queue_family_index) {
info.flags |= vk::DeviceQueueCreateFlags::INTERNALLY_SYNCHRONIZED_KHR;
}
unsafe { get2(device, &info, p_queue) };
let queue = unsafe { *p_queue };
unsafe { stamp(&ds, queue.as_raw() as *mut c_void) };
QUEUE_TO_DEVICE_KEY.insert(queue.as_raw(), key);
crate::state::QUEUE_TO_FAMILY.insert(queue.as_raw(), info.queue_family_index);
}
/// Enumerate device extensions supported by the physical device.
unsafe fn enumerate_device_extensions(
istate: &crate::dispatch::NextInstanceFn,
apps/nescapture/src/discovery.rs
+2 -2
View File
@@ -2,7 +2,7 @@
// discovery.rs — Phase 6: per-draw logging for shader hash discovery
//
// When HUDLESS_DISCOVER=1 is set, every vkCmdDraw* call is logged to:
// /tmp/hudless_discover_$EXE.log
// /tmp/nescapture_discover_$EXE.log
//
// Log format:
// frame=0001 draw=00042 vert=0x1a2b3c4d5e6f7890 frag=0xaabbccddeeff0011 verts=6 blend=true depth=false
@@ -40,7 +40,7 @@ fn ensure_log_file() -> bool {
.unwrap_or_else(|| "unknown".to_string())
});
let path = format!("/tmp/hudless_discover_{}.log", exe_name);
let path = format!("/tmp/nescapture_discover_{}.log", exe_name);
match OpenOptions::new().create(true).append(true).open(&path) {
Ok(f) => {
log::info!("discovery logging to {}", path);
apps/nescapture/src/dispatch.rs
+14 -45
View File
@@ -38,6 +38,8 @@ pub type PFN_vkDestroyDevice =
unsafe extern "system" fn(vk::Device, *const vk::AllocationCallbacks);
pub type PFN_vkGetDeviceQueue = unsafe extern "system" fn(vk::Device, u32, u32, *mut vk::Queue);
pub type PFN_vkGetDeviceQueue2 =
unsafe extern "system" fn(vk::Device, *const vk::DeviceQueueInfo2, *mut vk::Queue);
pub type PFN_vkQueuePresentKHR =
unsafe extern "system" fn(vk::Queue, *const vk::PresentInfoKHR) -> vk::Result;
@@ -190,20 +192,6 @@ pub type PFN_vkGetImageSubresourceLayout = unsafe extern "system" fn(
*mut vk::SubresourceLayout,
);
// DRM format modifiers. Both optional: without them the capture ring stays
// linear, which is what it was before it could be anything else.
pub type PFN_vkGetPhysicalDeviceFormatProperties2 = unsafe extern "system" fn(
vk::PhysicalDevice,
vk::Format,
*mut vk::FormatProperties2<'_>,
);
pub type PFN_vkGetImageDrmFormatModifierPropertiesEXT = unsafe extern "system" fn(
vk::Device,
vk::Image,
*mut vk::ImageDrmFormatModifierPropertiesEXT<'_>,
) -> vk::Result;
// Timestamp queries around the capture blit. All optional: without them the
// blit's GPU cost is simply not reported.
pub type PFN_vkGetPhysicalDeviceProperties =
@@ -219,21 +207,14 @@ pub type PFN_vkCreateQueryPool = unsafe extern "system" fn(
*mut vk::QueryPool,
) -> vk::Result;
pub type PFN_vkDestroyQueryPool = unsafe extern "system" fn(
vk::Device,
vk::QueryPool,
*const vk::AllocationCallbacks,
);
pub type PFN_vkDestroyQueryPool =
unsafe extern "system" fn(vk::Device, vk::QueryPool, *const vk::AllocationCallbacks);
pub type PFN_vkCmdResetQueryPool =
unsafe extern "system" fn(vk::CommandBuffer, vk::QueryPool, u32, u32);
pub type PFN_vkCmdWriteTimestamp = unsafe extern "system" fn(
vk::CommandBuffer,
vk::PipelineStageFlags,
vk::QueryPool,
u32,
);
pub type PFN_vkCmdWriteTimestamp =
unsafe extern "system" fn(vk::CommandBuffer, vk::PipelineStageFlags, vk::QueryPool, u32);
pub type PFN_vkGetQueryPoolResults = unsafe extern "system" fn(
vk::Device,
@@ -246,13 +227,6 @@ pub type PFN_vkGetQueryPoolResults = unsafe extern "system" fn(
vk::QueryResultFlags,
) -> vk::Result;
// DMA-BUF fd export (used to share final_image with pixelforge zero-copy)
pub type PFN_vkGetMemoryFdKHR = unsafe extern "system" fn(
vk::Device,
*const vk::MemoryGetFdInfoKHR,
*mut std::os::raw::c_int,
) -> vk::Result;
// ── Phase 4: Synchronisation ─────────────────────────────────────────────────
pub type PFN_vkCreateFence = unsafe extern "system" fn(
@@ -366,13 +340,14 @@ pub type PFN_vkResetCommandBuffer =
// ── Dispatch table structs ────────────────────────────────────────────────────
pub struct NextInstanceFn {
/// The instance itself, for building the encoder's view of it.
pub instance: vk::Instance,
/// The Vulkan version the application asked for, which bounds what core
/// functionality exists on its devices.
pub api_version: u32,
pub get_instance_proc_addr: PFN_vkGetInstanceProcAddr,
pub destroy_instance: PFN_vkDestroyInstance,
pub get_physical_device_memory_properties: PFN_vkGetPhysicalDeviceMemoryProperties,
/// `None` on an instance below Vulkan 1.1 without
/// `VK_KHR_get_physical_device_properties2`. Without it the modifier list
/// cannot be queried and the capture ring stays linear.
pub get_physical_device_format_properties2: Option<PFN_vkGetPhysicalDeviceFormatProperties2>,
/// Needed for `timestampPeriod`, which turns device ticks into nanoseconds.
pub get_physical_device_properties: Option<PFN_vkGetPhysicalDeviceProperties>,
/// Needed for a queue family's `timestampValidBits`. A family reporting
@@ -388,6 +363,9 @@ pub struct NextDeviceFn {
pub get_device_proc_addr: PFN_vkGetDeviceProcAddr,
pub destroy_device: PFN_vkDestroyDevice,
pub get_device_queue: PFN_vkGetDeviceQueue,
/// Core in 1.1. Needed to fetch queues created with flags, which is how a
/// queue shared with the encoder is created.
pub get_device_queue2: Option<PFN_vkGetDeviceQueue2>,
pub queue_present_khr: Option<PFN_vkQueuePresentKHR>,
// Phase 1
@@ -421,15 +399,6 @@ pub struct NextDeviceFn {
pub cmd_pipeline_barrier: PFN_vkCmdPipelineBarrier,
pub cmd_copy_image: PFN_vkCmdCopyImage,
pub get_image_subresource_layout: PFN_vkGetImageSubresourceLayout,
/// `None` when `VK_KHR_external_memory_fd` is unavailable.
/// Required for DMA-BUF export to pixelforge's VkDevice.
pub get_memory_fd_khr: Option<PFN_vkGetMemoryFdKHR>,
/// `None` when `VK_EXT_image_drm_format_modifier` was not enabled. The
/// driver picks the modifier from the list it is offered, so this is how
/// the layer learns which one it actually got — and the importer needs the
/// exact value, not the list.
pub get_image_drm_format_modifier_properties_ext:
Option<PFN_vkGetImageDrmFormatModifierPropertiesEXT>,
// Phase 4 — blit timing. All-or-nothing: the ring only times the blit when
// every one of these loaded and the presenting queue family can timestamp.
apps/nescapture/src/dmabuf_import.rs
-259
View File
@@ -1,259 +0,0 @@
//! DMA-BUF import support for zero-copy video encoding.
//!
//! This module provides the ability to import Linux DMA-BUF file descriptors as
//! Vulkan images for direct video encoding without CPU-side copies.
//!
//! `DmaBufImporter` caches imported Vulkan resources per compositor buffer index
//! so that pre-allocated GBM buffers are imported only once. Subsequent frames
//! from the same buffer reuse the cached `VkImage` and `VkDeviceMemory`,
//! eliminating per-frame Vulkan object creation and layout transitions.
use anyhow::Result;
use ash::vk;
use log::debug;
use pixelforge::VideoContext;
use std::os::fd::RawFd;
use std::os::unix::io::{BorrowedFd, IntoRawFd};
/// Information about a single DMA-BUF plane.
#[derive(Debug, Clone, Copy)]
pub struct DmaBufPlane {
/// File descriptor for the DMA-BUF.
pub fd: RawFd,
/// Offset within the DMA-BUF to the start of this plane.
pub offset: u32,
/// Row stride in bytes.
pub stride: u32,
/// DRM format modifier.
pub modifier: u64,
}
/// Cached Vulkan resources for a single compositor buffer slot.
struct CachedImport {
image: vk::Image,
memory: vk::DeviceMemory,
}
/// Importer for DMA-BUF file descriptors into Vulkan images.
///
/// Owns a per-buffer-index cache of `VkImage` + `VkDeviceMemory`.
/// Layout transitions are deferred to the consumer (e.g. `ColorConverter`)
/// to avoid a separate GPU submission per first-time import.
pub struct DmaBufImporter {
context: VideoContext,
external_memory_fd: ash::khr::external_memory_fd::Device,
/// Per-buffer-index cache. Index corresponds to `ExportedFrame::buffer_index`.
cached_imports: Vec<Option<CachedImport>>,
}
impl DmaBufImporter {
/// Create a new DMA-BUF importer.
pub fn new(context: VideoContext) -> Result<Self> {
let external_memory_fd =
ash::khr::external_memory_fd::Device::load(context.instance(), context.device());
Ok(Self {
context,
external_memory_fd,
cached_imports: Vec::new(),
})
}
/// Import a DMA-BUF as a Vulkan image, reusing a cached import when
/// the same `buffer_index` has been seen before.
///
/// The `format` parameter specifies the Vulkan format matching the DMA-BUF
/// pixel format (e.g. `B8G8R8A8_UNORM` for SDR, `A2B10G10R10_UNORM_PACK32`
/// for 10-bit HDR, `R16G16B16A16_SFLOAT` for FP16 HDR).
///
/// Returns `(image, needs_transition)` where `needs_transition` is `true`
/// for first-time imports whose image is still in `UNDEFINED` layout.
/// The caller is responsible for transitioning the image (e.g. by passing
/// the appropriate `src_layout` to `ColorConverter::convert`).
pub fn import_or_reuse(
&mut self,
buffer_index: usize,
width: u32,
height: u32,
format: vk::Format,
planes: &[DmaBufPlane],
) -> Result<(vk::Image, bool)> {
// Grow the cache vector if needed.
if self.cached_imports.len() <= buffer_index {
self.cached_imports.resize_with(buffer_index + 1, || None);
}
if let Some(cached) = &self.cached_imports[buffer_index] {
return Ok((cached.image, false));
}
// First time seeing this buffer — full import.
debug!(
"First import for buffer {buffer_index}: {}x{}, format={:?}, fd={}, stride={}, modifier={:#x}",
width, height, format, planes[0].fd, planes[0].stride, planes[0].modifier
);
let (image, memory) = self.import_internal(width, height, format, planes)?;
self.cached_imports[buffer_index] = Some(CachedImport { image, memory });
Ok((image, true))
}
/// Perform the raw Vulkan import of a DMA-BUF with the specified format.
///
/// Returns the `(VkImage, VkDeviceMemory)` pair. The image is in
/// `UNDEFINED` layout; the caller must transition it.
fn import_internal(
&self,
width: u32,
height: u32,
format: vk::Format,
planes: &[DmaBufPlane],
) -> Result<(vk::Image, vk::DeviceMemory)> {
if planes.is_empty() {
return Err(anyhow::anyhow!("At least one DMA-BUF plane is required"));
}
let device = self.context.device();
// Build DRM format modifier plane layouts for all planes.
// AMD modifiers (e.g. tiled/DCC) may require multiple planes;
// the layout count must match the modifier's expected plane count.
let plane_layouts: Vec<vk::SubresourceLayout> = planes
.iter()
.map(|p| {
vk::SubresourceLayout::default()
.offset(p.offset as u64)
.row_pitch(p.stride as u64)
})
.collect();
let modifier = planes[0].modifier;
let mut drm_format_modifier_info =
vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(modifier)
.plane_layouts(&plane_layouts);
let mut external_memory_info = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
external_memory_info.p_next = &mut drm_format_modifier_info
as *mut vk::ImageDrmFormatModifierExplicitCreateInfoEXT
as *mut _;
let mut image_create_info = vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(format)
.extent(vk::Extent3D {
width,
height,
depth: 1,
})
.mip_levels(1)
.array_layers(1)
.samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT)
.usage(vk::ImageUsageFlags::TRANSFER_SRC | vk::ImageUsageFlags::SAMPLED)
.sharing_mode(vk::SharingMode::EXCLUSIVE)
.initial_layout(vk::ImageLayout::UNDEFINED);
image_create_info.p_next =
&mut external_memory_info as *mut vk::ExternalMemoryImageCreateInfo as *mut _;
let image = unsafe { device.create_image(&image_create_info, None) }
.map_err(|e| anyhow::anyhow!("DMA-BUF image creation: {e}"))?;
// Memory requirements.
let mem_requirements = unsafe { device.get_image_memory_requirements(image) };
// FD memory properties.
let mut memory_fd_properties = vk::MemoryFdPropertiesKHR::default();
unsafe {
self.external_memory_fd.get_memory_fd_properties(
vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT,
planes[0].fd,
&mut memory_fd_properties,
)
}
.map_err(|e| anyhow::anyhow!("Failed to get memory FD properties: {e}"))?;
// Duplicate the FD — vkAllocateMemory consumes it.
let fd = unsafe { BorrowedFd::borrow_raw(planes[0].fd) }
.try_clone_to_owned()
.map_err(|e| anyhow::anyhow!("Failed to duplicate DMA-BUF FD: {e}"))?
.into_raw_fd();
let mut import_memory_fd_info = vk::ImportMemoryFdInfoKHR::default()
.handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT)
.fd(fd);
let memory_type_bits =
mem_requirements.memory_type_bits & memory_fd_properties.memory_type_bits;
debug!(
"Memory allocation: size={}, image_type_bits={:#x}, fd_type_bits={:#x}, combined={:#x}",
mem_requirements.size,
mem_requirements.memory_type_bits,
memory_fd_properties.memory_type_bits,
memory_type_bits
);
let memory_type_index = self
.context
.find_memory_type(memory_type_bits, vk::MemoryPropertyFlags::empty())
.ok_or_else(|| {
// The numbers, in the error rather than in the debug! above
// it: fd_type_bits=0 means the driver could not resolve the
// descriptor at all, which is a different fault from a
// mismatch, and the difference is the whole diagnosis.
anyhow::anyhow!(
"No suitable memory type for DMA-BUF import: \
image_type_bits={:#x} & fd_type_bits={:#x} = {:#x}, \
size={}, format={:?}, modifier={:#x}, stride={}",
mem_requirements.memory_type_bits,
memory_fd_properties.memory_type_bits,
memory_type_bits,
mem_requirements.size,
format,
planes[0].modifier,
planes[0].stride
)
})?;
// Dedicated allocation (required by many drivers for external memory).
let mut dedicated_alloc_info = vk::MemoryDedicatedAllocateInfo::default().image(image);
import_memory_fd_info.p_next =
&mut dedicated_alloc_info as *mut vk::MemoryDedicatedAllocateInfo as *mut _;
let mut alloc_info = vk::MemoryAllocateInfo::default()
.allocation_size(mem_requirements.size)
.memory_type_index(memory_type_index);
alloc_info.p_next = &mut import_memory_fd_info as *mut vk::ImportMemoryFdInfoKHR as *mut _;
let memory = unsafe { device.allocate_memory(&alloc_info, None) }.map_err(|e| {
unsafe { device.destroy_image(image, None) };
anyhow::anyhow!("DMA-BUF memory import: {e}")
})?;
if let Err(e) = unsafe { device.bind_image_memory(image, memory, 0) } {
unsafe {
device.free_memory(memory, None);
device.destroy_image(image, None);
}
return Err(anyhow::anyhow!("DMA-BUF memory bind: {e}"));
}
Ok((image, memory))
}
}
impl Drop for DmaBufImporter {
fn drop(&mut self) {
let device = self.context.device();
unsafe {
// Clean up cached imports.
for cached in self.cached_imports.drain(..).flatten() {
device.destroy_image(cached.image, None);
device.free_memory(cached.memory, None);
}
}
}
}
apps/nescapture/src/encode.rs
+2604 -356
View File
File diff suppressed because it is too large Load Diff
apps/nescapture/src/instance.rs
+58 -8
View File
@@ -41,7 +41,42 @@ pub unsafe extern "system" fn vkCreateInstance(
let next_create: PFN_vkCreateInstance =
unsafe { load_instance_fn(next_gipa, vk::Instance::null(), b"vkCreateInstance\0") };
let result = unsafe { next_create(p_create_info, p_allocator, p_instance) };
// The encoder needs Vulkan 1.1 on the game's instance to run on the game's
// device. An application asking for 1.0 is raised to 1.1 where the loader
// has it: 1.1 only adds to 1.0, so nothing the application can do behaves
// any differently, and without it the encoder needs a device of its own.
let asked = unsafe {
let app = (*p_create_info).p_application_info;
if app.is_null() {
vk::API_VERSION_1_0
} else {
(*app).api_version
}
};
let mut raised_app;
let mut raised_ci;
let mut create_info = p_create_info;
let mut api_version = asked;
if asked < vk::API_VERSION_1_1 && unsafe { loader_version(next_gipa) } >= vk::API_VERSION_1_1 {
raised_app = unsafe {
let app = (*p_create_info).p_application_info;
if app.is_null() {
vk::ApplicationInfo::default()
} else {
*app
}
};
raised_app.api_version = vk::API_VERSION_1_1;
raised_ci = unsafe { *p_create_info };
raised_ci.p_application_info = &raised_app;
create_info = &raised_ci;
api_version = vk::API_VERSION_1_1;
log::info!(
"instance asked for Vulkan 1.0; created as 1.1 so the encoder can share its devices"
);
}
let result = unsafe { next_create(create_info, p_allocator, p_instance) };
if result != vk::Result::SUCCESS {
return result;
}
@@ -51,6 +86,8 @@ pub unsafe extern "system" fn vkCreateInstance(
// ── Build and store per-instance dispatch table ───────────────────────────
let istate = Arc::new(NextInstanceFn {
instance,
api_version,
get_instance_proc_addr: next_gipa,
destroy_instance: unsafe { load_instance_fn(next_gipa, instance, b"vkDestroyInstance\0") },
get_physical_device_memory_properties: unsafe {
@@ -60,13 +97,6 @@ pub unsafe extern "system" fn vkCreateInstance(
b"vkGetPhysicalDeviceMemoryProperties\0",
)
},
get_physical_device_format_properties2: unsafe {
crate::try_load_instance_fn(
next_gipa,
instance,
b"vkGetPhysicalDeviceFormatProperties2\0",
)
},
get_physical_device_properties: unsafe {
crate::try_load_instance_fn(next_gipa, instance, b"vkGetPhysicalDeviceProperties\0")
},
@@ -85,6 +115,26 @@ pub unsafe extern "system" fn vkCreateInstance(
vk::Result::SUCCESS
}
/// The highest instance version the loader below supports; 1.0 when it cannot
/// say, since vkEnumerateInstanceVersion is itself a 1.1 addition.
unsafe fn loader_version(next_gipa: crate::dispatch::PFN_vkGetInstanceProcAddr) -> u32 {
type Enumerate = unsafe extern "system" fn(*mut u32) -> vk::Result;
let Some(f) = (unsafe {
crate::try_load_instance_fn::<Enumerate>(
next_gipa,
vk::Instance::null(),
b"vkEnumerateInstanceVersion\0",
)
}) else {
return vk::API_VERSION_1_0;
};
let mut version = vk::API_VERSION_1_0;
if unsafe { f(&mut version) } != vk::Result::SUCCESS {
return vk::API_VERSION_1_0;
}
version
}
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkDestroyInstance(
instance: vk::Instance,
apps/nescapture/src/lib.rs
+11 -4
View File
@@ -27,20 +27,20 @@ mod config;
mod device;
mod discovery;
mod dispatch;
mod dmabuf_import;
mod encode;
mod framebuffer;
mod instance;
mod memory;
mod modifiers;
mod pacing;
mod pipeline;
mod present;
mod rate_probe;
mod shader;
mod shared;
mod slots;
mod state;
mod timing;
mod swapchain;
mod timing;
use commands::{
vkCmdBeginRenderPass, vkCmdBeginRenderingKHR, vkCmdBindPipeline, vkCmdDraw, vkCmdDrawIndexed,
@@ -48,7 +48,7 @@ use commands::{
vkCmdDrawIndirect, vkCmdDrawIndirectCount, vkCmdDrawIndirectCountKHR, vkCmdEndRenderPass,
vkCmdEndRenderingKHR,
};
use device::{vkCreateDevice, vkDestroyDevice, vkGetDeviceQueue};
use device::{vkCreateDevice, vkDestroyDevice, vkGetDeviceQueue, vkGetDeviceQueue2};
use framebuffer::{
vkAllocateCommandBuffers, vkCreateFramebuffer, vkCreateImageView, vkDestroyFramebuffer,
vkDestroyImageView, vkFreeCommandBuffers,
@@ -100,10 +100,16 @@ pub(crate) struct VkLayerDeviceLink {
pfnNextGetDeviceProcAddr: Option<PFN_vkGetDeviceProcAddr>,
}
/// Stamps the loader's dispatch data into a dispatchable object a layer
/// created itself. See [`crate::device::stamp`].
pub(crate) type PFN_vkSetDeviceLoaderData =
unsafe extern "system" fn(vk::Device, *mut c_void) -> vk::Result;
#[repr(C)]
pub(crate) union VkLayerCreateInfoU {
pub pLayerInfo: *mut VkLayerInstanceLink,
pub pDeviceLayerInfo: *mut VkLayerDeviceLink,
pub pfnSetDeviceLoaderData: Option<PFN_vkSetDeviceLoaderData>,
}
const VK_STRUCTURE_TYPE_LOADER_INSTANCE_CREATE_INFO: i32 = 47;
@@ -271,6 +277,7 @@ unsafe fn match_device_fn(name: &[u8]) -> Option<RawFn> {
b"vkCreateDevice" => Some(to_raw(vkCreateDevice as *const () as usize)),
b"vkDestroyDevice" => Some(to_raw(vkDestroyDevice as *const () as usize)),
b"vkGetDeviceQueue" => Some(to_raw(vkGetDeviceQueue as *const () as usize)),
b"vkGetDeviceQueue2" => Some(to_raw(vkGetDeviceQueue2 as *const () as usize)),
b"vkQueuePresentKHR" => Some(to_raw(vkQueuePresentKHR as *const () as usize)),
b"vkCreateShaderModule" => Some(to_raw(vkCreateShaderModule as *const () as usize)),
apps/nescapture/src/memory.rs
+6 -1
View File
@@ -43,7 +43,12 @@ pub fn pick_memory_type(types: &[MemoryType], bits: u32, want: Want) -> Option<u
match want {
Want::DeviceLocal => (0..types.len())
.find(|&i| allowed(i) && types[i].flags.contains(vk::MemoryPropertyFlags::DEVICE_LOCAL))
.find(|&i| {
allowed(i)
&& types[i]
.flags
.contains(vk::MemoryPropertyFlags::DEVICE_LOCAL)
})
.or_else(|| (0..types.len()).find(|&i| allowed(i)))
.map(|i| i as u32),
Want::HostCoherent => (0..types.len())
apps/nescapture/src/modifiers.rs
-112
View File
@@ -1,112 +0,0 @@
// ─────────────────────────────────────────────────────────────────────────────
// modifiers.rs — choosing a DRM format modifier for the capture ring
//
// The importer has always been able to take a tiled buffer: `dmabuf_import.rs`
// builds `VkImageDrmFormatModifierExplicitCreateInfoEXT` with per-plane
// layouts and creates the image with `DRM_FORMAT_MODIFIER_EXT` tiling. Only
// the producer was linear — hard-coded `ImageTiling::LINEAR` and a `modifier`
// of zero passed down with every frame — so every capture detiled a full frame
// on the write and the encoder sampled a linear image on the read.
//
// Picking the modifier is the whole of the decision and it is pure, so it is
// here and tested rather than buried in an unsafe block.
// ─────────────────────────────────────────────────────────────────────────────
/// One entry of `VkDrmFormatModifierPropertiesListEXT`, already filtered to
/// modifiers whose tiling features cover both our write and the encoder's read.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ModifierProps {
pub modifier: u64,
pub plane_count: u32,
}
/// `DRM_FORMAT_MOD_LINEAR`.
pub const LINEAR: u64 = 0;
/// `DRM_FORMAT_MOD_INVALID`, which a driver may list and which means "let the
/// driver choose" — not something to ask for explicitly.
pub const INVALID: u64 = 0x00ff_ffff_ffff_ffff;
/// Pick the modifier to allocate the capture ring with.
///
/// Single-plane only, and deliberately so. A multi-plane modifier needs an
/// offset and a stride per plane on the import side, and the export path hands
/// out one fd with one stride — so accepting one would produce an image the far
/// side reads at the wrong offsets, which arrives at the right size and frame
/// rate carrying nonsense. Prefer any real tiled modifier; fall back to linear,
/// which is what the ring used before this existed and always works.
pub fn pick_modifier(candidates: &[ModifierProps]) -> Option<ModifierProps> {
let usable = |m: &&ModifierProps| m.plane_count == 1 && m.modifier != INVALID;
candidates
.iter()
.filter(usable)
.find(|m| m.modifier != LINEAR)
.or_else(|| candidates.iter().filter(usable).find(|m| m.modifier == LINEAR))
.copied()
}
#[cfg(test)]
mod tests {
use super::*;
const TILED: u64 = 0x0200_0000_0000_0001;
#[test]
fn a_tiled_modifier_beats_linear() {
let c = [
ModifierProps { modifier: LINEAR, plane_count: 1 },
ModifierProps { modifier: TILED, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, TILED);
}
/// Order must not decide it — the driver lists them in its own order.
#[test]
fn a_tiled_modifier_wins_from_either_position() {
let c = [
ModifierProps { modifier: TILED, plane_count: 1 },
ModifierProps { modifier: LINEAR, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, TILED);
}
/// A multi-plane modifier with one exported fd would be imported at the
/// wrong plane offsets and produce a corrupt frame rather than an error.
#[test]
fn multi_plane_modifiers_are_refused() {
let c = [
ModifierProps { modifier: 0x0200_0000_0000_0002, plane_count: 2 },
ModifierProps { modifier: LINEAR, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, LINEAR);
}
/// A multi-plane tiled modifier must not beat a single-plane linear one
/// just for being tiled.
#[test]
fn tiling_does_not_excuse_a_plane_count_we_cannot_export() {
let c = [
ModifierProps { modifier: TILED, plane_count: 4 },
ModifierProps { modifier: LINEAR, plane_count: 1 },
];
assert_eq!(pick_modifier(&c).unwrap().modifier, LINEAR);
}
/// `DRM_FORMAT_MOD_INVALID` is not a modifier to ask for.
#[test]
fn the_invalid_modifier_is_never_chosen() {
let c = [ModifierProps { modifier: INVALID, plane_count: 1 }];
assert_eq!(pick_modifier(&c), None);
}
#[test]
fn nothing_usable_is_none() {
let c = [ModifierProps { modifier: TILED, plane_count: 4 }];
assert_eq!(pick_modifier(&c), None);
}
#[test]
fn an_empty_list_is_none() {
assert_eq!(pick_modifier(&[]), None);
}
}
apps/nescapture/src/present.rs
+118 -40
View File
@@ -1,5 +1,5 @@
use crate::capture;
use crate::encode::{CapturedFrame, FrameSource, PipelineConfig, PipelineHandle};
use crate::encode::{CapturedFrame, FrameSource, PipelineConfig, PipelineHandle, PresentStep};
use crate::slots::SlotGuard;
use crate::state::{DEVICE_STATE, QUEUE_TO_DEVICE_KEY};
use ash::vk::{self, Handle};
@@ -32,6 +32,7 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
// it is the game's own frame time with this layer's cost excluded — the
// three spans then partition the wall clock between presents exactly.
let entered = std::time::Instant::now();
note_present(&ds, PresentStep::InHook);
ds.frame_counter.fetch_add(1, Ordering::Relaxed);
ds.hud_detected_frame.store(false, Ordering::Relaxed);
@@ -55,9 +56,8 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
// Rewriting the wait semaphores is only well defined for a single
// swapchain. A multi-swapchain present is rare enough that passing it
// through untouched beats getting the interposition subtly wrong.
let single_swapchain = pi.swapchain_count == 1
&& !pi.p_swapchains.is_null()
&& !pi.p_image_indices.is_null();
let single_swapchain =
pi.swapchain_count == 1 && !pi.p_swapchains.is_null() && !pi.p_image_indices.is_null();
let submission = if single_swapchain {
unsafe { try_capture(&ds, queue, pi) }
@@ -68,9 +68,11 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
let down_us = std::cell::Cell::new(std::time::Duration::ZERO);
let call_down = |info: *const vk::PresentInfoKHR| match ds.fp.queue_present_khr {
Some(f) => {
note_present(&ds, PresentStep::Presenting);
let t = std::time::Instant::now();
let r = unsafe { f(queue, info) };
down_us.set(t.elapsed());
note_present(&ds, PresentStep::InHook);
r
}
None => vk::Result::ERROR_EXTENSION_NOT_PRESENT,
@@ -82,7 +84,6 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
return r;
};
// The blit consumed the application's wait semaphores, so the present waits
// on ours instead. Presenting on the originals as well would be a second
// wait on an already-consumed signal.
@@ -121,11 +122,7 @@ pub unsafe extern "system" fn vkQueuePresentKHR(
/// `layer` is everything in this hook that is not the down-call, both sides of
/// it added together, so `gap + layer + down` accounts for the wall clock
/// between one present and the next with nothing unattributed.
fn finish(
ds: &crate::state::DeviceState,
entered: std::time::Instant,
down: std::time::Duration,
) {
fn finish(ds: &crate::state::DeviceState, entered: std::time::Instant, down: std::time::Duration) {
// Everything this hook cost, before any deliberate waiting.
let worked = std::time::Instant::now();
@@ -138,6 +135,7 @@ fn finish(
Err(_) => std::time::Duration::ZERO,
};
if !held.is_zero() {
note_present(ds, PresentStep::Holding);
std::thread::sleep(held);
}
@@ -167,12 +165,24 @@ fn finish(
if let Ok(mut last) = ds.last_present_return.lock() {
*last = Some(now);
}
note_present(ds, PresentStep::InGame);
}
/// Tell the stall watchdog where the game's present thread is. A no-op until
/// the pipeline exists.
pub fn note_present(ds: &crate::state::DeviceState, step: PresentStep) {
if let Ok(enc) = ds.encoder.lock()
&& let Some(ref h) = *enc
{
h.note_present(step);
}
}
/// Everything the present hook needs to carry from the blit to the worker.
struct Submission {
slot: SlotGuard,
present_wait: vk::Semaphore,
blit: Option<pixelforge::TimelinePoint>,
width: u32,
height: u32,
sc_fmt: vk::Format,
@@ -185,6 +195,13 @@ unsafe fn try_capture(
pi: &vk::PresentInfoKHR,
) -> Option<Submission> {
let image_index = unsafe { *pi.p_image_indices } as usize;
// An image acquired from a retired swapchain may still be presented, and
// the tracked images belong to its replacement: the same index there is a
// different image, possibly of a different size.
let presented = unsafe { *pi.p_swapchains };
if !crate::swapchain::is_current(*ds.swapchain.lock().ok()?, presented) {
return None;
}
let (sc_image, sc_fmt, sc_ext, image_count) = {
let images = ds.swapchain_images.lock().ok()?;
let fmt = *ds.swapchain_format.lock().ok()?;
@@ -196,7 +213,7 @@ unsafe fn try_capture(
};
// Gate before any GPU work is queued. A game presenting faster than the
// target would otherwise pay a full blit and DMA-BUF export for frames the
// target would otherwise pay a full blit for frames the
// encoder throws away moments later.
let present_time = std::time::Instant::now();
let admitted = match ds.frame_gate.lock() {
@@ -221,14 +238,14 @@ unsafe fn try_capture(
}
}
let app_waits: &[vk::Semaphore] = if pi.wait_semaphore_count == 0 || pi.p_wait_semaphores.is_null()
{
&[]
} else {
unsafe {
std::slice::from_raw_parts(pi.p_wait_semaphores, pi.wait_semaphore_count as usize)
}
};
let app_waits: &[vk::Semaphore] =
if pi.wait_semaphore_count == 0 || pi.p_wait_semaphores.is_null() {
&[]
} else {
unsafe {
std::slice::from_raw_parts(pi.p_wait_semaphores, pi.wait_semaphore_count as usize)
}
};
let submission = unsafe {
capture::capture_present_frame(
@@ -246,6 +263,7 @@ unsafe fn try_capture(
Some(Submission {
slot: submission.slot,
present_wait: submission.present_wait,
blit: submission.blit,
width: sc_ext.width,
height: sc_ext.height,
sc_fmt,
@@ -273,6 +291,7 @@ fn queue_for_encode(ds: &crate::state::DeviceState, submission: Submission) {
vk_colorspace: ds.swapchain_colorspace.load(Ordering::Relaxed),
present_time: submission.present_time,
slot: Some(submission.slot),
blit: submission.blit,
});
}
@@ -303,11 +322,31 @@ fn encoder_ready(ds: &crate::state::DeviceState, ds_key: usize, width: u32, heig
log::error!("no encode pipeline configuration; capture disabled");
return;
};
match PipelineHandle::new(cfg) {
// On the game's own device where it was created for that, and
// on a device of the encoder's own where it was not, or where
// adopting it fails.
let shared = ds.shared.as_ref().and_then(|s| match s.video_context() {
Ok(ctx) => Some(ctx),
Err(e) => {
log::warn!(
"could not encode on the game's device ({e}); using a device of its own"
);
None
}
});
let on_shared = shared.is_some();
match PipelineHandle::new(cfg, shared) {
Ok(h) => {
h.set_stall_probe(Box::new(move || describe_gpu(ds_key)));
// Before the handle is published: the first capture after
// it builds the ring, and must build it for this device.
ds.shared_active.store(on_shared, Ordering::Release);
if let Ok(mut enc) = ds.encoder.lock() {
*enc = Some(h);
}
if on_shared {
log::info!("encoding on the game's own device");
}
}
Err(e) => log::error!("encode pipeline: {e}"),
}
@@ -316,9 +355,13 @@ fn encoder_ready(ds: &crate::state::DeviceState, ds_key: usize, width: u32, heig
false
}
/// Wait for a frame's blit and turn its slot into something the encoder reads.
/// Turn a frame's slot into something the encoder reads.
///
/// This is the CPU handover the two devices need: pixelforge's `VkDevice`
/// On a device the encoder shares this is immediate: the slot's image is read
/// in place, and the encoder's GPU work waits on the blit's timeline point, so
/// nothing here waits at all.
///
/// Otherwise this is the CPU handover two devices need: pixelforge's `VkDevice`
/// shares no timeline with the game's, so no semaphore can bridge them and
/// somebody has to block. It used to be a thread of its own between the present
/// hook and the encoder; it is now the first thing the encoder thread does with
@@ -331,13 +374,19 @@ pub fn resolve_source(
) -> Option<FrameSource> {
let slot_index = frame.slot.as_ref()?.index();
if let Some(blit) = frame.blit {
let ring = ds.capture_ring.lock().ok()?;
let image = ring.as_ref()?.slots.get(slot_index)?.image;
return Some(FrameSource::Shared { image, blit });
}
// Copy the handles out and drop the ring lock before waiting: the present
// hook needs that lock every frame and must not queue behind a GPU wait.
let (fence, dmabuf_fd, stride, modifier, image, memory) = {
let (fence, image, memory) = {
let ring = ds.capture_ring.lock().ok()?;
ring.as_ref()
.and_then(|r| r.slots.get(slot_index))
.map(|s| (s.fence, s.dmabuf_fd, s.stride, s.modifier, s.image, s.memory))?
.map(|s| (s.fence, s.image, s.memory))?
};
let waited = unsafe { (ds.fp.wait_for_fences)(ds.raw, 1, &fence, vk::TRUE, 1_000_000_000) };
@@ -347,30 +396,59 @@ pub fn resolve_source(
}
// After the fence, so the queries have landed and `WAIT` returns at once.
if let Some(ns) = unsafe { capture::blit_gpu_time_ns(ds, slot_index) }
if let Some(ns) = unsafe { capture::blit_gpu_time_ns(ds, slot_index, true) }
&& let Ok(enc) = ds.encoder.lock()
&& let Some(ref h) = *enc
{
h.timing.blit.record(std::time::Duration::from_nanos(ns));
}
if dmabuf_fd >= 0 {
let duped = unsafe { libc::dup(dmabuf_fd) };
if duped < 0 {
log::warn!("dup of capture DMA-BUF failed — frame dropped");
return None;
}
return Some(FrameSource::DmaBuf {
fd: duped,
stride,
// The slot's own modifier. This was hard-coded to zero, which was
// true only because the producer could not ask for anything else.
modifier,
});
}
match unsafe { capture::read_frame_pixels(ds, image, memory, frame.width, frame.height) } {
Some(p) if !p.is_empty() => Some(FrameSource::Pixels(p)),
_ => None,
}
}
/// Whether the GPU has got through the capture work this layer gave it, for
/// the stall watchdog.
///
/// A blit is queued behind the game's frame on the game's own queue, so a blit
/// that never completes means that queue stopped: the game is waiting on the
/// GPU. One that did complete means the GPU finished everything in front of
/// it, and whatever the game is waiting for is not GPU work on that queue.
///
/// Never blocks. It runs exactly when something may be holding a lock forever.
fn describe_gpu(ds_key: usize) -> String {
let Some(ds) = DEVICE_STATE.get(&ds_key).map(|s| s.clone()) else {
return "device gone".into();
};
let slots = format!(
"{} of {} capture slots free",
ds.capture_slots.available(),
crate::state::CAPTURE_SLOTS
);
let Ok(ring) = ds.capture_ring.try_lock() else {
return format!("{slots}; capture ring locked");
};
let Some(ring) = ring.as_ref() else {
return format!("{slots}; no capture ring");
};
if let Some(shared) = ds.shared.as_ref()
&& !ring.blit_timeline.is_null()
{
let reached = shared
.counter(ring.blit_timeline)
.map_or("unknown".to_string(), |v| v.to_string());
return format!(
"{slots}; last blit submitted signals {}, the GPU has reached {reached}",
ring.blit_value
);
}
let pending = ring
.slots
.iter()
.filter(|s| unsafe { (ds.fp.wait_for_fences)(ds.raw, 1, &s.fence, vk::TRUE, 0) }
== vk::Result::TIMEOUT)
.count();
format!("{slots}; {pending} blits submitted and not yet complete")
}
apps/nescapture/src/rate_probe.rs
+415
View File
@@ -0,0 +1,415 @@
//! How long the encoder takes to actually reach a bitrate it was told to use.
//!
//! # Why this exists
//!
//! The bitrate controller decides how often it is worth deciding. Everything
//! else about its design follows from one number nobody has measured: the time
//! between `set_target_bitrate` and the encoder actually producing that rate.
//!
//! Simulated against a measured 1000-mile path, the difference is the whole
//! design. With the encoder settling in a quarter second, a controller sampling
//! the send queue five times a second holds the queue at 50 ms. With the
//! encoder taking a second, the same controller at any rate holds it at about
//! 1400 ms, which is unplayable, and sampling faster buys nothing at all --
//! there is no point reacting quicker than the thing being steered can move.
//!
//! So this measures it, rather than picking a control rate and hoping.
//!
//! # Shape
//!
//! Self-driving on purpose. The ladder and the dwell are fixed here so that two
//! people on two networks produce numbers that can be laid beside each other;
//! if the steps came from a person moving a slider, they would not be.
//!
//! While a sweep is running the encoder ignores bitrates from anywhere else.
//! A controller adjusting in the background would be a second hand on the same
//! dial, and the measurement would describe the argument rather than the
//! encoder.
use std::time::{Duration, Instant};
/// Targets to step through, in kbps.
///
/// Large steps and small ones, downward and upward. Downward matters most --
/// that is the direction taken under congestion, and the direction where being
/// slow costs latency rather than picture -- but an encoder can easily be quick
/// one way and slow the other, so both are here.
const LADDER: [u32; 6] = [6_000, 1_500, 6_000, 3_000, 1_000, 4_000];
/// How long to sit at each rung.
///
/// Long enough to settle and then be seen to be steady. If settling turns out
/// to take longer than this, that is itself the answer and it is reported as a
/// failure to settle rather than as a number.
const DWELL: Duration = Duration::from_secs(5);
/// Window over which the produced rate is measured.
///
/// Short enough to see a change quickly, long enough that one large frame does
/// not look like a rate. At sixty frames a second this averages about thirty.
const WINDOW: Duration = Duration::from_millis(500);
/// How close counts as arrived.
const TOLERANCE: f32 = 0.10;
/// How long it must stay inside the tolerance to count as settled, rather than
/// having passed through on the way somewhere else.
const HOLD: Duration = Duration::from_millis(500);
/// What one rung of the ladder turned out to cost.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct StepReport {
pub from_kbps: u32,
pub to_kbps: u32,
/// Milliseconds from the command to the produced rate first being inside
/// the tolerance and staying there. `None` if it never settled.
pub settle_ms: Option<u32>,
pub settle_frames: Option<u32>,
/// Produced rate over the last second of the dwell, as a fraction of the
/// target. This is the overshoot the controller has to divide out: a
/// hardware encoder asked for 1000 does not produce 1000.
pub steady_ratio: f32,
/// Keyframes during the rung.
pub keyframes: u32,
/// The largest keyframe seen, in bytes.
///
/// Reported rather than folded in, because it is a different quantity with
/// a different consumer. Settling is about the rate control finding its
/// operating point; a keyframe is a single burst handed to the transport
/// whole. The controller needs both and must not confuse them: measured at
/// 4 s GOP, a run where keyframes were counted in the rate said settling
/// took 2266 ms where the same encoder with no keyframes said 450 ms, and
/// in one case the number went *down* when keyframes were added. That is
/// not an encoder being erratic, it is a window catching an IDR.
pub keyframe_bytes: u32,
/// How long that keyframe alone occupies the link at this rung's target,
/// in milliseconds. This is the burst a queue has to absorb, and the
/// reason a queue setpoint cannot simply be set below it.
pub keyframe_ms: u32,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Phase {
/// Letting the encoder reach a steady state before the first step.
WarmUp,
Stepping,
Done,
}
pub struct RateProbe {
phase: Phase,
rung: usize,
/// When the current rung was commanded.
began: Instant,
current_kbps: u32,
/// `(when, bytes)` inside the measurement window, delta frames only.
///
/// Keyframes are deliberately absent. One IDR is worth many delta frames,
/// so a half-second window containing one reports a rate several times the
/// truth, leaves the tolerance band, and restarts the settle clock -- which
/// measures the GOP rather than the encoder.
samples: Vec<(Instant, u32)>,
keyframe_bytes: u32,
/// Frames since the current rung was commanded.
frames: u32,
keyframes: u32,
/// When the produced rate first entered the tolerance, if it still is.
inside_since: Option<Instant>,
settled: Option<(u32, u32)>,
/// The last second of the dwell, for the steady-state ratio.
steady: Vec<(Instant, u32)>,
reports: Vec<StepReport>,
}
impl RateProbe {
/// A probe that has not started stepping yet.
pub fn new(now: Instant, starting_kbps: u32) -> Self {
Self {
phase: Phase::WarmUp,
rung: 0,
began: now,
current_kbps: starting_kbps,
samples: Vec::new(),
keyframe_bytes: 0,
frames: 0,
keyframes: 0,
inside_since: None,
settled: None,
steady: Vec::new(),
reports: Vec::new(),
}
}
pub fn finished(&self) -> bool {
self.phase == Phase::Done
}
/// Whether a bitrate from elsewhere should be ignored.
pub fn owns_the_bitrate(&self) -> bool {
self.phase != Phase::Done
}
/// One encoded frame.
pub fn observe(&mut self, now: Instant, bytes: u32, keyframe: bool) {
if keyframe {
self.keyframe_bytes = self.keyframe_bytes.max(bytes);
} else {
self.samples.push((now, bytes));
}
self.samples
.retain(|(t, _)| now.duration_since(*t) <= WINDOW);
// The steady-state ratio *does* include keyframes: it answers what the
// link actually carries for a given target, which is the whole output.
self.steady.push((now, bytes));
self.steady
.retain(|(t, _)| now.duration_since(*t) <= Duration::from_secs(1));
if self.phase != Phase::Stepping {
return;
}
self.frames += 1;
if keyframe {
self.keyframes += 1;
}
if self.settled.is_some() {
return;
}
let Some(rate) = self.measured_kbps(now) else {
return;
};
let drift =
(rate as f32 - self.current_kbps as f32).abs() / self.current_kbps.max(1) as f32;
if drift > TOLERANCE {
// Left the band, so whatever it was doing was not settling.
self.inside_since = None;
return;
}
let entered = *self.inside_since.get_or_insert(now);
if now.duration_since(entered) >= HOLD {
// Credit the moment it arrived, not the moment it had stayed long
// enough to prove it: the hold is evidence about the arrival, not
// part of the time the encoder took.
let ms = entered.duration_since(self.began).as_millis() as u32;
self.settled = Some((ms, self.frames));
}
}
/// Produced rate over the window, or `None` before there is a window's worth.
fn measured_kbps(&self, now: Instant) -> Option<u32> {
let oldest = self.samples.first()?.0;
let span = now.duration_since(oldest);
if span < WINDOW / 2 {
return None;
}
let bits: u64 = self.samples.iter().map(|(_, b)| u64::from(*b) * 8).sum();
Some((bits as f64 / span.as_secs_f64() / 1000.0) as u32)
}
/// The next target to apply, when the current rung is done.
pub fn due_step(&mut self, now: Instant) -> Option<u32> {
if now.duration_since(self.began) < DWELL {
return None;
}
if self.phase == Phase::Stepping {
self.close_rung(now);
}
if self.rung >= LADDER.len() {
self.phase = Phase::Done;
return None;
}
let next = LADDER[self.rung];
self.rung += 1;
self.phase = Phase::Stepping;
self.began = now;
self.frames = 0;
self.keyframes = 0;
self.keyframe_bytes = 0;
self.inside_since = None;
self.settled = None;
let from = self.current_kbps;
self.current_kbps = next;
let _ = from;
Some(next)
}
fn close_rung(&mut self, now: Instant) {
let steady_bits: u64 = self.steady.iter().map(|(_, b)| u64::from(*b) * 8).sum();
let span = self
.steady
.first()
.map(|(t, _)| now.duration_since(*t).as_secs_f64())
.unwrap_or(0.0);
let steady_kbps = if span > 0.0 {
steady_bits as f64 / span / 1000.0
} else {
0.0
};
let from = self.reports.last().map_or(0, |r| r.to_kbps);
self.reports.push(StepReport {
from_kbps: from,
to_kbps: self.current_kbps,
settle_ms: self.settled.map(|(ms, _)| ms),
settle_frames: self.settled.map(|(_, f)| f),
steady_ratio: (steady_kbps / f64::from(self.current_kbps.max(1))) as f32,
keyframes: self.keyframes,
keyframe_bytes: self.keyframe_bytes,
keyframe_ms: (u64::from(self.keyframe_bytes) * 8 / u64::from(self.current_kbps.max(1)))
as u32,
});
}
pub fn reports(&self) -> &[StepReport] {
&self.reports
}
/// The number the controller design turns on: the slowest settle observed.
///
/// The slowest rather than the average, for the same reason the bitrate
/// controller reads the worst client's report: a loop that keeps up with
/// the typical step and not the worst one is a loop that falls behind
/// exactly when the path is changing, which is the only time it matters.
pub fn worst_settle_ms(&self) -> Option<u32> {
self.reports.iter().map(|r| r.settle_ms).max().flatten()
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Feed a probe frames at `kbps` for `secs`, sixty a second.
fn feed(probe: &mut RateProbe, start: Instant, secs: f32, kbps: u32) -> Instant {
let frames = (secs * 60.0) as u32;
let bytes = (u64::from(kbps) * 1000 / 8 / 60) as u32;
let mut now = start;
for i in 1..=frames {
// From the start each time: stepping by a rounded frame interval
// drifts, and a helper that loses 4% of every second makes a probe
// measuring seconds look broken when it is not.
now = start + Duration::from_secs_f64(f64::from(i) / 60.0);
probe.observe(now, bytes, false);
}
now
}
#[test]
fn an_encoder_that_changes_at_once_settles_at_once() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
let step = p.due_step(now).expect("first rung");
assert_eq!(step, 6_000);
now = feed(&mut p, now, 5.1, 6_000);
assert_eq!(p.due_step(now), Some(1_500));
// Produces the new rate immediately.
feed(&mut p, now, 2.0, 1_500);
let r = p.reports();
assert!(!r.is_empty());
assert!(
r[0].settle_ms.is_some_and(|ms| ms < 1_200),
"an instant encoder reported {:?}",
r[0].settle_ms
);
}
/// The case that would kill the fast-loop design, and must be visible.
#[test]
fn an_encoder_that_never_gets_there_reports_no_settle() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
p.due_step(now);
now = feed(&mut p, now, 5.1, 6_000);
assert_eq!(p.due_step(now), Some(1_500));
// Ignores the command completely.
now = feed(&mut p, now, 5.1, 6_000);
p.due_step(now);
let r = p.reports();
let step = r
.iter()
.find(|r| r.to_kbps == 1_500)
.expect("the 1500 rung");
assert_eq!(
step.settle_ms, None,
"an encoder that ignored us looked settled"
);
assert!(
step.steady_ratio > 3.0,
"steady ratio {} did not show it producing four times the target",
step.steady_ratio
);
}
/// Overshoot is the other number the controller needs, so it must be real.
#[test]
fn steady_ratio_reports_the_overshoot() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
p.due_step(now);
now = feed(&mut p, now, 5.1, 6_000);
p.due_step(now);
// Asked for 1500, produces 1875: the 25% overshoot measured on hardware.
now = feed(&mut p, now, 5.1, 1_875);
p.due_step(now);
let step = p.reports().iter().find(|r| r.to_kbps == 1_500).unwrap();
assert!(
(step.steady_ratio - 1.25).abs() < 0.1,
"steady ratio {} should be about 1.25",
step.steady_ratio
);
}
/// The confound that made a 4 s GOP look like a three-times slower
/// encoder, including making one rung appear *faster* when keyframes were
/// added -- which no encoder does, and which gave the measurement away.
#[test]
fn a_keyframe_landing_mid_window_does_not_delay_the_reported_settle() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = feed(&mut p, t0, 5.1, 6_000);
p.due_step(now);
now = feed(&mut p, now, 5.1, 6_000);
assert_eq!(p.due_step(now), Some(1_500));
// Settles immediately, but an IDR worth a second of bitrate lands in
// the middle of the window that is meant to prove it.
let bytes = (1_500u64 * 1000 / 8 / 60) as u32;
let base = now;
for i in 1..=320u32 {
now = base + Duration::from_secs_f64(f64::from(i) / 60.0);
let key = i == 40;
p.observe(now, if key { 1_500 * 1000 / 8 } else { bytes }, key);
}
p.due_step(now);
let step = p.reports().iter().find(|r| r.to_kbps == 1_500).unwrap();
assert!(
step.settle_ms.is_some_and(|ms| ms < 1_000),
"an encoder that settled at once reported {:?} because of one keyframe",
step.settle_ms
);
// And the burst is still reported, because the queue has to absorb it.
assert_eq!(step.keyframes, 1);
assert!(
step.keyframe_ms >= 900,
"a keyframe worth a second of bitrate reported {} ms",
step.keyframe_ms
);
}
#[test]
fn a_sweep_ends_and_does_not_step_forever() {
let t0 = Instant::now();
let mut p = RateProbe::new(t0, 6_000);
let mut now = t0;
for _ in 0..(LADDER.len() + 2) {
now = feed(&mut p, now, 5.1, 3_000);
p.due_step(now);
}
assert!(p.finished());
assert!(
!p.owns_the_bitrate(),
"a finished sweep still holds the dial"
);
assert_eq!(p.reports().len(), LADDER.len());
}
}
apps/nescapture/src/shared.rs
+1165
View File
File diff suppressed because it is too large Load Diff
apps/nescapture/src/slots.rs
+1 -1
View File
@@ -2,7 +2,7 @@
// slots.rs — ownership of the capture ring's destination buffers
//
// A captured frame travels from the present hook, through the capture worker,
// into the encoder thread, and its DMA-BUF must not be written again until the
// into the encoder thread, and its image must not be written again until the
// encoder has finished reading it. Tracking that by hand across three threads
// is how the single-buffer version got it wrong. Instead the slot index is
// carried by a guard that returns it to the pool when it drops, wherever that
apps/nescapture/src/state.rs
+43 -20
View File
@@ -25,21 +25,10 @@ pub const CAPTURE_SLOTS: usize = 4;
pub struct CaptureSlot {
pub image: vk::Image,
pub memory: vk::DeviceMemory,
/// Exported once at allocation and duplicated per frame. -1 if the export
/// failed, which sends that frame down the CPU readback path instead.
pub dmabuf_fd: std::os::raw::c_int,
pub stride: u32,
/// The DRM format modifier the driver gave this slot's image.
///
/// Carried per slot rather than assumed, and passed to the importer, which
/// creates its side with this exact value. It used to be hard-coded to
/// `DRM_FORMAT_MOD_LINEAR` on both sides — true at the time, because the
/// producer only ever asked for linear.
pub modifier: u64,
/// Signalled when this slot's blit has finished reading the swapchain and
/// writing the slot. The capture worker waits on it before handing the
/// DMA-BUF to the encoder, which reads it from a different VkDevice and so
/// cannot be synchronised with a semaphore.
/// writing the slot. Where the encoder has a device of its own, the encoder
/// thread waits on it before reading the slot back on the CPU; a device of
/// its own shares no timeline with the game's.
pub fence: vk::Fence,
}
@@ -75,12 +64,14 @@ pub struct CaptureRing {
pub timestamp_period: f32,
/// The extent `blits` were recorded for.
///
/// The ring is kept when the swapchain shrinks — `ensure_capture_ring`
/// accepts a ring at least as large as the request — so the extent can
/// change under a ring that is not rebuilt, and the recordings have to
/// follow it even though the images do not.
/// Kept separate from `size` because a swapchain can be recreated at the
/// same extent — on a format or present-mode change — which invalidates the
/// recordings without invalidating the images.
pub blit_extent: vk::Extent2D,
pub size: (u32, u32, vk::Format),
/// Which ring this is, counting from the device's first. For the log, so a
/// resize shows up as the ring it caused.
pub generation: u64,
/// Queue family the command pool was created for. Command buffers may only
/// be submitted to a queue of the family their pool belongs to, so a
/// present arriving on a different family rebuilds the ring rather than
@@ -102,6 +93,15 @@ pub struct CaptureRing {
/// cannot reacquire it until the present that waited on this semaphore is
/// done.
pub present_wait: Vec<vk::Semaphore>,
/// Timeline semaphore every blit signals, one value higher each time, or
/// null when the encoder does not share this device.
///
/// On a shared device this is the whole handover: the frame carries the
/// value its blit signals, and the encoder's GPU work waits on it. Nothing
/// waits on the CPU.
pub blit_timeline: vk::Semaphore,
/// The value the last blit signalled.
pub blit_value: u64,
}
/// Index into [`CaptureRing::blits`] for one (swapchain image, slot) pair.
@@ -177,6 +177,17 @@ pub struct DeviceState {
pub raw: vk::Device,
pub physical_device: vk::PhysicalDevice,
pub fp: NextDeviceFn,
/// The encoder's view of this device, when it runs on it. `None` when the
/// device could not be created with what the encoder needs, in which case
/// the encoder uses a device of its own.
pub shared: Option<crate::shared::SharedDevice>,
/// Whether the encode pipeline really runs on `shared`. Set once the
/// pipeline has been built on it; until then, and for good if that fails,
/// capture works as it does for a device of the encoder's own.
pub shared_active: std::sync::atomic::AtomicBool,
/// The loader's callback for stamping dispatchable objects this layer
/// creates itself. See [`crate::device::stamp`].
pub set_loader_data: Option<crate::PFN_vkSetDeviceLoaderData>,
// Phase 1: shader / pipeline
pub shader_registry: DashMap<u64, u64>,
@@ -197,7 +208,7 @@ pub struct DeviceState {
pub hudless_memory: std::sync::Mutex<Option<vk::DeviceMemory>>,
pub hudless_size: std::sync::Mutex<(u32, u32, vk::Format)>,
// Phase 4: final-frame capture (DMA-BUF exportable)
// Phase 4: final-frame capture
pub capture_ring: std::sync::Mutex<Option<CaptureRing>>,
/// Which ring slots are free. Held separately from the ring itself so a
/// slot can be returned from the encoder thread without taking the lock
@@ -219,6 +230,9 @@ pub struct DeviceState {
pub swapchain_transfer_src: std::sync::atomic::AtomicBool,
pub frame_counter: std::sync::atomic::AtomicU64,
/// Rings built for this device so far. Names the current one; see
/// [`CaptureRing::generation`].
pub ring_generation: std::sync::atomic::AtomicU64,
// Phase 3/4: per-frame HUD detection flags
pub hud_detected_frame: std::sync::atomic::AtomicBool,
@@ -229,7 +243,6 @@ pub struct DeviceState {
// Phase 7: encode + IPC pipeline (lazy-init on first frame)
pub encoder: std::sync::Mutex<Option<PipelineHandle>>,
// ── Frame-rate throttle ───────────────────────────────────────────
/// Decides which presented frames are worth capturing. Consulted in the
/// present hook, before any GPU work is queued, so a dropped frame costs
@@ -252,6 +265,16 @@ pub struct DeviceState {
pub encoder_starting: std::sync::atomic::AtomicBool,
}
impl DeviceState {
/// The shared device, when the encoder is actually running on it.
pub fn shared_encoder(&self) -> Option<&crate::shared::SharedDevice> {
self.shared.as_ref().filter(|_| {
self.shared_active
.load(std::sync::atomic::Ordering::Acquire)
})
}
}
// ── Per-command-buffer state ──────────────────────────────────────────────────
#[derive(Default, Clone)]
apps/nescapture/src/swapchain.rs
+51 -3
View File
@@ -125,6 +125,13 @@ pub unsafe extern "system" fn vkCreateSwapchainKHR(
vk::Result::SUCCESS
}
/// Whether `swapchain` is the one this layer tracks, which is the one most
/// recently created. A swapchain retired through `oldSwapchain` stays valid
/// until destroyed, and calls on it must not touch the tracked state.
pub fn is_current(tracked: Option<vk::SwapchainKHR>, swapchain: vk::SwapchainKHR) -> bool {
swapchain != vk::SwapchainKHR::null() && tracked == Some(swapchain)
}
#[unsafe(no_mangle)]
pub unsafe extern "system" fn vkDestroySwapchainKHR(
device: vk::Device,
@@ -133,8 +140,17 @@ pub unsafe extern "system" fn vkDestroySwapchainKHR(
) {
let key = unsafe { dispatch_key(device.as_raw() as *const c_void) };
if let Some(ds) = DEVICE_STATE.get(&key) {
*ds.swapchain.lock().unwrap() = None;
*ds.swapchain_images.lock().unwrap() = Vec::new();
// Only the swapchain being tracked. A game recreating through
// `oldSwapchain` destroys the retired one after its replacement is
// created and its images fetched, and clearing then would leave the
// live swapchain with no images: every present after would skip
// capture, silently, until the next recreation.
let mut current = ds.swapchain.lock().unwrap();
if is_current(*current, swapchain) {
*current = None;
*ds.swapchain_images.lock().unwrap() = Vec::new();
}
drop(current);
if let Some(destroy_fn) = ds.fp.destroy_swapchain_khr {
unsafe { destroy_fn(device, swapchain, p_allocator) };
}
@@ -171,7 +187,8 @@ pub unsafe extern "system" fn vkGetSwapchainImagesKHR(
return result;
}
if !p_swapchain_images.is_null() {
// Images of a retired swapchain are not the ones capture reads.
if !p_swapchain_images.is_null() && is_current(*ds.swapchain.lock().unwrap(), swapchain) {
let count = unsafe { *p_swapchain_image_count as usize };
let images = unsafe { std::slice::from_raw_parts(p_swapchain_images, count) };
*ds.swapchain_images.lock().unwrap() = images.to_vec();
@@ -231,9 +248,11 @@ pub unsafe extern "system" fn vkAcquireNextImageKHR(
return vk::Result::ERROR_EXTENSION_NOT_PRESENT;
};
crate::present::note_present(&ds, crate::encode::PresentStep::Acquiring);
let started = std::time::Instant::now();
let result = unsafe { acquire(device, swapchain, timeout, semaphore, fence, p_image_index) };
record_acquire(&ds, started.elapsed());
crate::present::note_present(&ds, crate::encode::PresentStep::InGame);
result
}
@@ -250,8 +269,37 @@ pub unsafe extern "system" fn vkAcquireNextImage2KHR(
return vk::Result::ERROR_EXTENSION_NOT_PRESENT;
};
crate::present::note_present(&ds, crate::encode::PresentStep::Acquiring);
let started = std::time::Instant::now();
let result = unsafe { acquire(device, p_acquire_info, p_image_index) };
record_acquire(&ds, started.elapsed());
crate::present::note_present(&ds, crate::encode::PresentStep::InGame);
result
}
#[cfg(test)]
mod tests {
use super::*;
fn sc(raw: u64) -> vk::SwapchainKHR {
vk::SwapchainKHR::from_raw(raw)
}
#[test]
fn the_tracked_swapchain_is_current() {
assert!(is_current(Some(sc(2)), sc(2)));
}
#[test]
fn a_retired_swapchain_is_not() {
// Created 2 with oldSwapchain = 1; destroying 1 afterwards must leave
// 2's state alone.
assert!(!is_current(Some(sc(2)), sc(1)));
}
#[test]
fn nothing_is_current_once_the_tracked_one_is_gone() {
assert!(!is_current(None, sc(1)));
assert!(!is_current(None, vk::SwapchainKHR::null()));
}
}
apps/nescope/Cargo.toml
-2
View File
@@ -28,8 +28,6 @@ smithay = { version = "0.7", default-features = false, features = [
# Wayland client – connects to the host compositor to forward buffers.
wayland-client = "0.31"
wayland-protocols = { version = "0.32", features = ["client", "staging", "server"] }
# Needed to generate the gamescope_swapchain protocol bindings.
wayland-scanner = "0.31"
wayland-backend = "0.3"
# Event loop
apps/nescope/build.rs
-3
View File
@@ -1,3 +0,0 @@
fn main() {
println!("cargo:rerun-if-changed=src/protocols/gamescope-swapchain.xml");
}
apps/nescope/src/handlers.rs
+25 -2
View File
@@ -11,6 +11,7 @@ use smithay::desktop::Window;
use smithay::input::pointer::{CursorImageStatus, PointerHandle};
use smithay::input::{Seat, SeatHandler, SeatState};
use smithay::output::Output;
use smithay::reexports::wayland_protocols::xdg::shell::server::xdg_toplevel;
use smithay::reexports::wayland_server::protocol::wl_buffer;
use smithay::reexports::wayland_server::protocol::wl_output::WlOutput;
use smithay::reexports::wayland_server::protocol::wl_seat::WlSeat;
@@ -119,7 +120,6 @@ impl CompositorHandler for NescopeState {
fn destroyed(&mut self, surface: &WlSurface) {
self.hdr.surface_destroyed(surface);
self.vulkan_surfaces.remove(surface);
}
}
@@ -138,7 +138,7 @@ impl DmabufHandler for NescopeState {
_dmabuf: Dmabuf,
notifier: ImportNotifier,
) {
// Accept unconditionally — libhudless reads buffers
// Accept unconditionally — the nescapture layer reads buffers
// directly from the game's Vulkan queue; nescope doesn't need to.
let _ = notifier.successful::<NescopeState>();
}
@@ -187,6 +187,29 @@ impl XdgShellHandler for NescopeState {
self.determine_and_apply_focus();
}
// Granted, not merely acknowledged. The default answers with a configure
// that lacks the fullscreen state, which a client reads as a refusal: Wine
// then asks again on every window update and never treats its window as
// fullscreen, so a game switching to exclusive fullscreen -- Control does
// this on leaving its title screen -- stalls in the transition and stays
// where it was. Wine also scales an emulated display mode up to the output
// only for a fullscreen window.
fn fullscreen_request(&mut self, surface: ToplevelSurface, _output: Option<WlOutput>) {
surface.with_pending_state(|state| {
state.states.set(xdg_toplevel::State::Fullscreen);
state.size = Some((self.width as i32, self.height as i32).into());
});
surface.send_configure();
}
// Still the size of the output: there is nowhere else for a window to be.
fn unfullscreen_request(&mut self, surface: ToplevelSurface) {
surface.with_pending_state(|state| {
state.states.unset(xdg_toplevel::State::Fullscreen);
});
surface.send_configure();
}
fn new_popup(&mut self, _: PopupSurface, _: PositionerState) {}
fn grab(&mut self, _: PopupSurface, _: WlSeat, _: Serial) {}
fn reposition_request(&mut self, _: PopupSurface, _: PositionerState, _: u32) {}
apps/nescope/src/hdr.rs
+564 -286
View File
File diff suppressed because it is too large Load Diff
apps/nescope/src/input.rs
-20
View File
@@ -106,11 +106,6 @@ pub fn process_input(event: InputEvent, state: &mut NescopeState) {
_ => state.last_pointer_activity = std::time::Instant::now(),
}
// One-time X11 focus reset when the gamescope WSI surface is active.
if state.override_surface.is_some() && state.x11_focus_needs_reset {
state.sync_x11_focus();
}
match event {
InputEvent::KeyDown { keycode } => {
if let Some(kb) = state.seat.get_keyboard() {
@@ -363,21 +358,6 @@ fn clamp_cursor(state: &mut NescopeState) {
/// Find the focused target under the current cursor position.
pub fn surface_under(state: &NescopeState) -> Option<(KeyboardFocusTarget, Point<f64, Logical>)> {
if state.override_surface.is_some() {
if let Some(wid) = state.focused_x11_window {
for window in state.space.elements() {
if let Some(x11) = window.x11_surface() {
if x11.window_id() == wid {
let loc = state.space.element_geometry(window)?.loc;
return Some((KeyboardFocusTarget::Window(window.clone()), loc.to_f64()));
}
}
}
}
let (window, loc) = state.space.element_under(state.cursor_position)?;
return Some((KeyboardFocusTarget::Window(window.clone()), loc.to_f64()));
}
// Try element_under first
if let Some((window, loc)) = state.space.element_under(state.cursor_position) {
return Some((KeyboardFocusTarget::Window(window.clone()), loc.to_f64()));
apps/nescope/src/main.rs
+44 -28
View File
@@ -4,7 +4,7 @@
//!
//! nescope creates a virtual Wayland output, starts XWayland, and gives games
//! a complete compositor environment. Frames are captured externally by a
//! Vulkan interception library (`hudless`); nescope itself
//! Vulkan interception library (`nescapture`); nescope itself
//! never allocates a GBM pool or forwards DMA-BUFs.
//!
//! # Usage
@@ -17,7 +17,7 @@
//! --height <N> Output height [default: 1080]
//! --fps <N> Virtual refresh rate, advertised only [default: 60]
//! --frame-callback-hz <N> wl_surface.frame cadence [default: 1000]
//! --hdr Enable HDR protocols (wp_color_management_v1 + gamescope_swapchain)
//! --hdr Enable HDR colour management (wp_color_manager_v1)
//! --socket <NAME> Wayland socket name [default: nescope-0]
//! ```
//!
@@ -67,7 +67,6 @@ mod hdr;
mod input;
mod input_ipc;
mod libinput_backend;
mod protocols;
//mod screenshot_ipc;
//mod screenshot_wire;
mod state;
@@ -80,6 +79,22 @@ use state::{CalloopData, ClientState, NescopeState};
// CLI
// ---------------------------------------------------------------------------
/// A flag that can also arrive as an environment variable.
///
/// `--hdr` on its own still means true. The difference is what a value may be:
/// clap's own bool parser takes `true` and `false` and nothing else, so
/// `NESCOPE_HDR=1` -- which is how every other environment variable in this
/// stack is written, and the first thing anyone tries -- was rejected outright.
fn flag_value(value: &str) -> Result<bool, String> {
match value.trim().to_ascii_lowercase().as_str() {
"1" | "true" | "yes" | "on" => Ok(true),
"0" | "false" | "no" | "off" | "" => Ok(false),
other => Err(std::format!(
"expected 1 or 0 (true/false, yes/no and on/off are also taken), got {other:?}"
)),
}
}
#[derive(Parser, Debug)]
#[command(
name = "nescope",
@@ -125,8 +140,15 @@ struct Args {
#[arg(long, default_value = "1000", env = "NESCOPE_FRAME_CALLBACK_HZ")]
frame_callback_hz: u32,
/// Enable HDR protocols (wp_color_management_v1 + gamescope_swapchain_factory_v2).
#[arg(long, env = "NESCOPE_HDR")]
/// Enable HDR colour management (`wp_color_manager_v1`).
#[arg(
long,
env = "NESCOPE_HDR",
num_args = 0..=1,
default_value_t = false,
default_missing_value = "true",
value_parser = flag_value,
)]
hdr: bool,
/// Run XWayland, for Linux-native software with no Wayland support.
@@ -137,7 +159,14 @@ struct Args {
/// which is what the launch environment does -- and HDR is only offered on
/// the Wayland surface, so a game routed through XWayland loses it too.
/// Turn this on for the shrinking set of X11-only native software.
#[arg(long, env = "NESCOPE_XWAYLAND")]
#[arg(
long,
env = "NESCOPE_XWAYLAND",
num_args = 0..=1,
default_value_t = false,
default_missing_value = "true",
value_parser = flag_value,
)]
xwayland: bool,
/// Wayland socket name (created in $XDG_RUNTIME_DIR).
@@ -422,8 +451,7 @@ fn main() {
// wrong way round, and it capped them at 60 while sessions asked for 120.
// The capture layer holds the game instead, and this runs fast enough to
// stay out of the way.
let frame_interval =
Duration::from_micros(1_000_000 / args.frame_callback_hz.max(1) as u64);
let frame_interval = Duration::from_micros(1_000_000 / args.frame_callback_hz.max(1) as u64);
loop_handle
.insert_source(Timer::from_duration(frame_interval), move |_, _, data| {
if let Some(ref mut li) = data.libinput {
@@ -437,7 +465,7 @@ fn main() {
// ── CalloopData ───────────────────────────────────────────────────────
let socket_name_for_cleanup = args.socket.clone();
let command = args.command.clone();
let gamescope_wayland_socket = args.socket.clone();
let wayland_socket = args.socket.clone();
// ── libinput backend ─────────────────────────────────────────────────
let libinput_ctx =
@@ -491,7 +519,7 @@ fn main() {
// Put the game in its own process group so we can
// kill the whole tree at once with kill(-pgid, …).
.process_group(0)
.env("WAYLAND_DISPLAY", &gamescope_wayland_socket);
.env("WAYLAND_DISPLAY", &wayland_socket);
// DISPLAY only if XWayland is actually running. Setting it
// otherwise points clients at a server that is not there,
@@ -516,24 +544,6 @@ fn main() {
// this. Without it neither DX11 nor DX12 (vkd3d-proton
// through DXVK's dxgi) sees HDR as available.
cmd.env("DXVK_HDR", "1");
// Left set, but deliberately without ENABLE_GAMESCOPE_WSI
// alongside it, so it is inert unless somebody opts in.
//
// That pair activates gamescope's WSI layer, which
// predates Wayland colour management and works by
// hiding HDR from the driver and reporting it to the
// compositor out of band. We do not want it: it needs a
// layer this image does not ship, it only helps the
// XWayland path, and capture reads the colour space it
// hides -- measured, a game asking for HDR10 through it
// has its PQ samples encoded and tagged BT.709 SDR.
// Enabling it would trade no HDR for wrong HDR.
tracing::debug!(
gamescope_wayland_socket,
"HDR: Wayland colour management; gamescope WSI not enabled"
);
cmd.env("GAMESCOPE_WAYLAND_DISPLAY", &gamescope_wayland_socket);
}
// Detect GPU vendor from render device and set VK_DRIVER_FILES
@@ -625,6 +635,12 @@ fn main() {
}
}
// Answer any colour-management information requests that came in
// this iteration. Deferred to here because the event that ends
// them destroys the object, and doing that inside the request that
// created it panics the backend -- see .
data.state.hdr.flush_information();
// ── Flush Wayland clients ─────────────────────────────────
if let Err(e) = data.display.flush_clients() {
tracing::warn!("Error flushing Wayland clients: {e}");
apps/nescope/src/protocols/gamescope-swapchain.xml
-194
View File
@@ -1,194 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="gamescope_swapchain">
<copyright>
Copyright © 2023 Joshua Ashton for Valve Software
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
</copyright>
<description summary="gamescope-specific swapchain protocol">
This is a private Gamescope protocol. Regular Wayland clients must not use
it.
</description>
<interface name="gamescope_swapchain_factory_v2" version="1">
<request name="destroy" type="destructor"></request>
<request name="create_swapchain">
<description summary="create Gamescope swapchain interface">
</description>
<arg name="surface" type="object" interface="wl_surface"
summary="target surface"/>
<arg name="callback" type="new_id" interface="gamescope_swapchain"
summary="new swapchain object"/>
</request>
</interface>
<interface name="gamescope_swapchain" version="1">
<request name="destroy" type="destructor"></request>
<request name="override_window_content">
<description summary="override an X11's window wl_surface">
Xwayland creates a wl_surface for each X11 window. It sends a
WL_SURFACE_ID client message to indicate the mapping between the X11
windows and the wl_surface objects.
This request overrides this mapping for a given X11 window, allowing an
X11 client to submit buffers via the Wayland protocol. The override
only affects buffer submission. Everything else (e.g. input events)
still uses Xwayland's WL_SURFACE_ID.
x11_server is gotten by the GAMESCOPE_XWAYLAND_SERVER_ID property on the
root window of the associated server.
</description>
<arg name="gamescope_xwayland_server_id" type="uint" summary="gamescope xwayland server ID"/>
<arg name="x11_window" type="uint" summary="X11 window ID"/>
</request>
<request name="swapchain_feedback">
<description summary="provide swapchain feedback">
Provide swapchain feedback to the compositor.
This is what the useless tearing protocol should have been.
Absolutely not enough information in the final protocol to do what we want for SteamOS --
which is have the Allow Tearing toggle apply to *both* Mailbox + Immediate and NOT fifo,
essentially acting as an override for tearing on/off for games.
The upstream protocol is very useless for our usecase here.
Provides image count ahead of time instead of needing to try and calculate it from
an initial stall if we are doing low latency.
Provides colorspace info for us to do HDR for both HDR10 PQ and scRGB.
The upstream HDR efforts seem to have no interest in supporting scRGB but we *need* that so /shrug
We can do it here now! Yipee!
Swapchain feedback solves so many problems! :D
</description>
<arg name="image_count" type="uint" summary="image count of swapchain"/>
<arg name="vk_format" type="uint" summary="VkFormat of swapchain"/>
<arg name="vk_colorspace" type="uint" summary="VkColorSpaceKHR of swapchain"/>
<arg name="vk_composite_alpha" type="uint" summary="VkCompositeAlphaFlagBitsKHR of swapchain"/>
<arg name="vk_pre_transform" type="uint" summary="VkSurfaceTransformFlagBitsKHR of swapchain"/>
<arg name="vk_clipped" type="uint" summary="clipped (VkBool32) of swapchain"/>
<arg name="vk_engine_name" type="string" summary="Engine name"/>
</request>
<request name="set_present_mode">
<description summary="Add a fifo queue constraint"/>
<arg name="vk_present_mode" type="uint" summary="VkPresentModeKHR"/>
</request>
<request name="set_hdr_metadata">
<description summary="set HDR metadata for a surface">
Forward HDR metadata from Vulkan to the compositor.
HDR Metadata Infoframe as per CTA 861.G spec.
This is expected to match exactly with the spec.
display_primary_*:
Color Primaries of the Data.
Specifies X and Y coordinates.
These are coded as unsigned 16-bit values in units of
0.00002, where 0x0000 represents zero and 0xC350
represents 1.0000.
white_point_*:
White Point of Colorspace Data.
Specifies X and Y coordinates.
These are coded as unsigned 16-bit values in units of
0.00002, where 0x0000 represents zero and 0xC350
represents 1.0000.
max_display_mastering_luminance:
Max Mastering Display Luminance.
This value is coded as an unsigned 16-bit value in units of 1 cd/m2,
where 0x0001 represents 1 cd/m2 and 0xFFFF represents 65535 cd/m2.
max_display_mastering_luminance:
Min Mastering Display Luminance.
This value is coded as an unsigned 16-bit value in units of
0.0001 cd/m2, where 0x0001 represents 0.0001 cd/m2 and 0xFFFF
represents 6.5535 cd/m2.
max_cll:
Max Content Light Level.
This value is coded as an unsigned 16-bit value in units of 1 cd/m2,
where 0x0001 represents 1 cd/m2 and 0xFFFF represents 65535 cd/m2.
max_fall:
Max Frame Average Light Level.
This value is coded as an unsigned 16-bit value in units of 1 cd/m2,
where 0x0001 represents 1 cd/m2 and 0xFFFF represents 65535 cd/m2.
</description>
<arg name="display_primary_red_x" type="uint" summary="red primary x coordinate"/>
<arg name="display_primary_red_y" type="uint" summary="red primary y coordinate"/>
<arg name="display_primary_green_x" type="uint" summary="green primary x coordinate"/>
<arg name="display_primary_green_y" type="uint" summary="green primary y coordinate"/>
<arg name="display_primary_blue_x" type="uint" summary="blue primary x coordinate"/>
<arg name="display_primary_blue_y" type="uint" summary="blue primary y coordinate"/>
<arg name="white_point_x" type="uint" summary="white point x coordinate"/>
<arg name="white_point_y" type="uint" summary="white point y coordinate"/>
<arg name="max_display_mastering_luminance" type="uint" summary="max display mastering luminance"/>
<arg name="min_display_mastering_luminance" type="uint" summary="min display mastering luminance"/>
<arg name="max_cll" type="uint" summary="max content light level"/>
<arg name="max_fall" type="uint" summary="max frame average light level"/>
</request>
<request name="set_present_time">
<description summary="display timing of next commit">
Sets the display timing of the next commit.
This gets reset to 0s in the compositor's state after a commit.
</description>
<arg name="present_id" type="uint" summary="application provided presentation id"/>
<arg name="desired_present_time_hi" type="uint" summary="high part of the desired presentation time for this commit. Uses CLOCK_MONOTONIC. 0 = present as soon as possible."/>
<arg name="desired_present_time_lo" type="uint" summary="low part of the desired presentation time for this commit. Uses CLOCK_MONOTONIC. 0 = present as soon as possible."/>
</request>
<event name="past_present_timing">
<description summary="information about past presentation">
Gives information on the past presentation timing
</description>
<arg name="present_id" type="uint" summary="application provided presentation id"/>
<arg name="desired_present_time_hi" type="uint" summary="high part of the desired presentation time for the commit. (from the app)"/>
<arg name="desired_present_time_lo" type="uint" summary="low part of the desired presentation time for the commit. (from the app)"/>
<arg name="actual_present_time_hi" type="uint" summary="high part of the actual present time for this commit."/>
<arg name="actual_present_time_lo" type="uint" summary="low part of the actual present time for this commit."/>
<arg name="earliest_present_time_hi" type="uint" summary="high part of the refresh time that Gamescope thought this commit was done by."/>
<arg name="earliest_present_time_lo" type="uint" summary="low part of the refresh time that Gamescope thought this commit was done by."/>
<arg name="present_margin_hi" type="uint" summary="high part of the difference between earliest present time and the earliest latch time"/>
<arg name="present_margin_lo" type="uint" summary="low part of the difference between earliest present time and the earliest latch time"/>
</event>
<event name="refresh_cycle">
<description summary="information about refresh cycle for this swapchain">
Gives information on the refresh cycle for this swapchain
</description>
<arg name="refresh_cycle_hi" type="uint" summary="high part of the refresh cycle in nanos"/>
<arg name="refresh_cycle_lo" type="uint" summary="low part of the refresh cycle in nanos"/>
</event>
<event name="retired">
<description summary="Swapchain was remotely retired"></description>
</event>
</interface>
</protocol>
apps/nescope/src/protocols/mod.rs
-16
View File
@@ -1,16 +0,0 @@
//! Generated Wayland protocol bindings for the gamescope swapchain protocol.
#![allow(non_upper_case_globals, non_camel_case_types, unused)]
use smithay::reexports::wayland_server;
use wayland_server::protocol::*;
pub mod __interfaces {
use super::wayland_server;
use wayland_server::backend as wayland_backend;
use wayland_server::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("src/protocols/gamescope-swapchain.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_server_code!("src/protocols/gamescope-swapchain.xml");
apps/nescope/src/state.rs
+27 -41
View File
@@ -24,6 +24,7 @@ use calloop::channel::Sender;
use smithay::desktop::utils::{
OutputPresentationFeedback, send_frames_surface_tree,
surface_presentation_feedback_flags_from_states, surface_primary_scanout_output,
take_presentation_feedback_surface_tree,
};
use smithay::desktop::{Space, Window};
use smithay::input::pointer::CursorImageStatus;
@@ -96,13 +97,18 @@ impl ClientData for ClientState {
// X11 atoms + connection
// ---------------------------------------------------------------------------
/// Gamescope-compatible X11 atoms used for focus and HDR signalling.
/// Gamescope-compatible X11 atoms, for focus.
///
/// Focus only. HDR used to be signalled through one of these as well, back
/// when a WSI layer inside the game reported the colour space out of band;
/// that route is gone and `wp_color_manager_v1` carries it. These remain
/// because Steam reads them to work out which window it is looking at, and
/// that is unrelated to colour.
pub struct CachedAtoms {
pub net_active_window: u32,
pub gamescope_focused_app: u32,
pub gamescope_focusable_apps: u32,
pub gamescope_focusable_windows: u32,
pub gamescope_hdr_output_feedback: u32,
pub gamescope_xwayland_server_id: u32,
pub xa_window: u32,
pub xa_cardinal: u32,
@@ -157,10 +163,6 @@ pub struct NescopeState {
pub focused_app_id: u32,
/// True when X11 focus needs to be re-synced on the next input event.
pub x11_focus_needs_reset: bool,
/// Gamescope WSI override surface (direct Vulkan → Wayland bypass).
pub override_surface: Option<WlSurface>,
/// Surfaces that have announced themselves as Vulkan via gamescope protocol.
pub vulkan_surfaces: HashSet<WlSurface>,
// ── Input ─────────────────────────────────────────────────────────────
/// Sender half of the input channel — clone and hand to callers.
@@ -267,7 +269,7 @@ impl NescopeState {
let (dmabuf_state, dmabuf_global) =
build_dmabuf_global::<Self>(&display_handle, render_device.as_deref());
// HDR + gamescope swapchain globals (optional).
// Colour management, when asked for.
let hdr_state = HdrState::new(&display_handle, hdr);
// Input channel — the Sender is returned to the caller.
@@ -309,8 +311,6 @@ impl NescopeState {
focused_x11_window: None,
focused_app_id: 0,
x11_focus_needs_reset: false,
override_surface: None,
vulkan_surfaces: HashSet::new(),
input_tx: input_tx.clone(),
cursor_position: Point::from((0.0f64, 0.0f64)),
cursor_status: CursorImageStatus::default_named(),
@@ -398,10 +398,6 @@ impl NescopeState {
gamescope_focused_app: intern_atom(&conn, b"GAMESCOPE_FOCUSED_APP"),
gamescope_focusable_apps: intern_atom(&conn, b"GAMESCOPE_FOCUSABLE_APPS"),
gamescope_focusable_windows: intern_atom(&conn, b"GAMESCOPE_FOCUSABLE_WINDOWS"),
gamescope_hdr_output_feedback: intern_atom(
&conn,
b"GAMESCOPE_HDR_OUTPUT_FEEDBACK",
),
gamescope_xwayland_server_id: intern_atom(
&conn,
b"GAMESCOPE_XWAYLAND_SERVER_ID",
@@ -422,8 +418,10 @@ impl NescopeState {
}
}
/// Write gamescope-specific X11 root window properties so the WSI layer
/// can discover this compositor as a gamescope-compatible server.
/// Write the gamescope-compatible X11 root window properties.
///
/// These say which application has focus and what could take it, which is
/// what Steam looks for. Nothing here concerns colour.
pub fn set_gamescope_atoms(
&self,
conn: &smithay::reexports::x11rb::rust_connection::RustConnection,
@@ -438,14 +436,6 @@ impl NescopeState {
let replace = PropMode::REPLACE;
let cardinal = AtomEnum::CARDINAL;
// HDR output feedback — set to 1 when HDR is active.
let _ = conn.change_property32(
replace,
root,
atoms.gamescope_hdr_output_feedback,
cardinal,
&[1u32],
);
// XWayland server ID — always 0 for a standalone compositor.
let _ = conn.change_property32(
replace,
@@ -465,16 +455,6 @@ impl NescopeState {
tracing::debug!("Set gamescope atoms on display :{display_number}");
}
// -----------------------------------------------------------------------
// Override surface (gamescope WSI bypass)
// -----------------------------------------------------------------------
/// Register the gamescope WSI override surface for an X11 window.
pub fn override_window_surface(&mut self, x11_window: u32, surface: WlSurface) {
tracing::debug!(x11_window, "Registered gamescope WSI override surface");
self.override_surface = Some(surface);
}
// -----------------------------------------------------------------------
// Resize
// -----------------------------------------------------------------------
@@ -769,12 +749,24 @@ impl NescopeState {
// 1. Release the held buffer → frees a swapchain image for the game.
self.held_buffer.take();
// 2. Presentation feedback — tell clients about vsync timing.
// 2. Presentation feedback — every frame committed since the last tick
// is reported presented, on the one output there is.
//
// Not filtered by primary scan-out output: that is recorded by a
// renderer, and nothing here renders, so the filter matched no
// surface ever. Feedback then resolved only as `discarded`, when the
// next commit superseded it -- which never happens for a client that
// waits for its last present before drawing the next. A Vulkan
// client with present-wait under FIFO does exactly that: Control on
// VKD3D-Proton froze on leaving its title screen, GPU idle, the game
// still running behind a stream that no longer moved.
let mut output_presentation_feedback = OutputPresentationFeedback::new(&output);
let on_output =
|_: &WlSurface, _: &smithay::wayland::compositor::SurfaceData| Some(output.clone());
for window in self.space.elements().cloned().collect::<Vec<_>>() {
window.take_presentation_feedback(
&mut output_presentation_feedback,
surface_primary_scanout_output,
on_output,
|_, _| wp_presentation_feedback::Kind::Vsync,
);
}
@@ -797,12 +789,6 @@ impl NescopeState {
}
}
if let Some(ref s) = self.override_surface {
send_frames_surface_tree(s, &output, now, Some(Duration::ZERO), |_, _| {
Some(output.clone())
});
}
// 4. Send periodic stats over IPC
let now = std::time::Instant::now();
if now.duration_since(self.last_stats_time) >= std::time::Duration::from_secs(1) {
apps/neshub/Cargo.toml
+1 -1
View File
@@ -11,7 +11,7 @@ name = "neshub"
path = "src/main.rs"
[dependencies]
iroh = "1.1.0"
iroh = "1.2"
anyhow.workspace = true
clap.workspace = true
apps/neshub/src/control.rs
+914
View File
@@ -0,0 +1,914 @@
//! Deciding what the encoder may spend.
//!
//! # Why this exists at all
//!
//! Nothing used to decide. The encoder took a bitrate from an environment
//! variable nobody set, so every session offered the same 10 Mbps whatever the
//! path between the two ends could carry. On a path sustaining under 3 Mbps that
//! does not degrade the picture, it removes it: most fragments are dropped,
//! every frame made of several fragments arrives incomplete, and the keyframe --
//! the largest frame there is -- never completes either, so nothing ever
//! recovers.
//!
//! # Why the receiver decides, and not this end
//!
//! The obvious inputs are here: the hub holds the QUIC connection and can read
//! its round-trip time, its congestion window and its remaining datagram buffer.
//! They were measured, over an entire multi-session run, saying the path was
//! healthy while the client was receiving almost nothing -- 182 ms round trip,
//! *zero* QUIC packet loss, and arrivals falling from 1776 to 239 datagrams a
//! second.
//!
//! Part of that is structural rather than bad luck. `send_datagram` evicts the
//! oldest queued datagrams and returns `Ok`, so a sender that is overrunning the
//! path is told nothing at all; the "frames dropped" counter it feeds cannot be
//! non-zero however badly things are going. So the primary evidence is the
//! receiver's own report of what arrived, and this end's view is the fallback
//! for when no report has come in.
//!
//! # Shape
//!
//! Slow and dull on purpose. This is not a congestion controller in the QUIC
//! sense and does not try to be: it finds the right order of magnitude for a
//! video bitrate and stays there, at one decision a second. The transport
//! underneath is still doing real congestion control on its own packets.
use nesprotocol::{ControlMode, ReceiverReport};
/// Loss above this means the path is being overrun and the target comes down.
const LOSS_DECREASE: f32 = 0.10;
/// Loss below this means there is room to climb.
const LOSS_INCREASE: f32 = 0.02;
/// What to keep of the measured goodput when backing off.
///
/// Backing off to *below* what actually got through, rather than to a fraction
/// of what we were asking for, is what makes recovery quick. At 10 Mbps into a
/// 3 Mbps path the next target is about 2.5 Mbps rather than 8.5, so one step
/// does what twenty multiplicative decreases would.
const BACKOFF: f32 = 0.85;
/// How much of the ceiling to add per calm second. Twenty steps from floor to
/// ceiling: slow enough that a brief quiet spell does not undo a back-off.
const CLIMB_FRACTION: u32 = 20;
/// Don't actuate for less than this fraction of the current target.
///
/// Retuning is cheap now -- it no longer rebuilds the encoder or forces a
/// keyframe -- but a line in the log every second saying the bitrate moved by
/// 1% is noise that hides the changes that matter.
const DEADBAND: f32 = 0.10;
/// Never go below this, whatever the ceiling is.
///
/// Under this a 1080p stream is not worth sending and the right answer is fewer
/// pixels rather than fewer bits, which is a decision about the tier and not one
/// this can make. It also has to leave room for audio, which shares the
/// connection and is not counted against the video ceiling.
const ABSOLUTE_FLOOR_KBPS: u32 = 800;
/// Seconds without a receiver report before this end's own view is used instead.
const SILENT_TICKS_BEFORE_FALLBACK: u32 = 3;
/// Backlog, in milliseconds of video, that means the path is being overrun.
///
/// Not a tuned number: it is a latency budget. At sixty frames a second a frame
/// is under 17 ms, so a quarter second of backlog is fifteen frames already
/// handed over and not yet gone -- a player is looking at a picture from before
/// their last four keypresses. There is no bitrate worth that, so past this the
/// target comes down whatever the loss says.
const QUEUE_DECREASE_MS: u32 = 250;
/// Backlog, in milliseconds, under which the path is considered clear.
///
/// Climbing needs a stronger warrant than holding does, because climbing is
/// what digs the queue. Five frames or so of backlog is the most that can be
/// outstanding and still be called live.
const QUEUE_CLIMB_MS: u32 = 80;
/// The band a target must stay inside.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Limits {
pub ceiling_kbps: u32,
}
impl Limits {
pub fn new(ceiling_kbps: u32) -> Self {
Self { ceiling_kbps }
}
/// The lowest this may go.
pub fn floor_kbps(&self) -> u32 {
ABSOLUTE_FLOOR_KBPS
.max(self.ceiling_kbps / 10)
// A ceiling below the floor is a tier nobody should have configured,
// but clamping the wrong way round would put the target *above* the
// ceiling, which is the one thing a ceiling must never allow.
.min(self.ceiling_kbps)
}
fn clamp(&self, kbps: u32) -> u32 {
kbps.clamp(self.floor_kbps(), self.ceiling_kbps)
}
}
/// What this end can see about the path, for when the far end is not talking.
///
/// Every field is optional because every one of them can be unavailable: a
/// connection with no established path has no round-trip time, and a congestion
/// window is only meaningful once something has been sent.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct PathView {
/// Congestion window in bytes, as the transport currently believes it.
pub cwnd_bytes: Option<u64>,
/// Round-trip time in milliseconds.
pub rtt_ms: Option<u32>,
/// Datagram bytes handed to the transport that have not yet left.
///
/// The backlog is the one thing this end can see that says the path is
/// being overrun *while it is still only late*. Loss says so too, but only
/// afterwards, and on a path that queues rather than drops, "afterwards"
/// can be several seconds.
pub backlog_bytes: Option<u64>,
}
impl PathView {
/// How long the backlog takes to drain at `drain_kbps`, in milliseconds.
///
/// Bits divided by kilobits-per-second is milliseconds. The rate to divide
/// by is the one the queue actually drains at -- what is getting through --
/// and not what we are asking for, which is the number that is too high
/// whenever this matters.
pub fn backlog_ms(&self, drain_kbps: u32) -> Option<u32> {
let backlog = self.backlog_bytes?;
let drain = u64::from(drain_kbps.max(1));
Some((backlog.saturating_mul(8) / drain).min(u64::from(u32::MAX)) as u32)
}
/// Delivery rate the window and round trip imply, in kbps.
///
/// A window is a quantity of bytes in flight for one round trip, so the two
/// together are a rate. Treated as an estimate of last resort: it describes
/// the transport's own opinion of the path, and that opinion is exactly what
/// was observed to be wrong when this mattered most.
pub fn estimate_kbps(&self) -> Option<u32> {
let (cwnd, rtt) = (self.cwnd_bytes?, self.rtt_ms?.max(1));
let bits_per_second = cwnd.saturating_mul(8) * 1000 / u64::from(rtt);
Some((bits_per_second / 1000).min(u64::from(u32::MAX)) as u32)
}
}
/// Why the target is what it is, for the overlay and the log.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[repr(u8)]
pub enum Reason {
/// Loss was high; backed off to under what was actually getting through.
Congested,
/// Loss was in the middle band; left alone.
Holding,
/// Loss was low; climbing towards the ceiling.
Climbing,
/// No receiver report recently; using this end's view of the path.
Fallback,
/// No report and no usable view either; decaying towards the floor.
Blind,
/// Somebody set the bitrate by hand.
Manual,
/// The encoder is not under a bitrate at all, so there is nothing to decide.
ConstantQuality,
/// Nobody is connected, so there is no path to have an opinion about.
NoClients,
/// This second's numbers describe frames the hub chose not to send, so they
/// say nothing about the path.
SelfInflicted,
/// Nothing was lost, but the send queue is standing deep enough that the
/// picture is arriving late. Backed off on the queue rather than on loss.
Backlogged,
}
/// The controller's whole state.
#[derive(Debug, Clone, Copy)]
pub struct Controller {
/// The ceiling the box was given, which a client may lower but never raise.
///
/// Kept apart from `limits`, which holds whatever is in force *now*. Folding
/// the two together means a client that lowers the ceiling can never raise
/// it again, because the only number left to compare against is the one it
/// just lowered.
box_ceiling_kbps: u32,
/// What the controller currently believes the bitrate should be.
///
/// Kept apart from `sent_kbps` deliberately. Folding the two together looks
/// simpler and stalls the climb: the step up is a fraction of the *ceiling*,
/// so once the target passes half of it each step is under the deadband, and
/// a target that only moved when it actuated could never accumulate past
/// that point. Belief moves every second; only saying so is rationed.
target_kbps: u32,
/// What the encoder was last told, or `None` before anything was sent.
sent_kbps: Option<u32>,
limits: Limits,
mode: ControlMode,
/// Set when the encoder is in constant-QP; there is no bitrate to control.
constant_quality: bool,
silent_ticks: u32,
reason: Reason,
/// Last measured send-queue depth, kept for reporting rather than control.
backlog_ms: u32,
}
impl Controller {
/// Start at the ceiling.
///
/// Optimistic on purpose: the common case is a path that can carry the tier
/// that was sold, and a session that opened at its floor and climbed would
/// take twenty seconds to look like what was paid for. The first bad report
/// undoes it in one step.
pub fn new(limits: Limits) -> Self {
Self {
box_ceiling_kbps: limits.ceiling_kbps,
target_kbps: limits.ceiling_kbps,
sent_kbps: None,
limits,
mode: ControlMode::Auto,
constant_quality: false,
silent_ticks: 0,
reason: Reason::Holding,
backlog_ms: 0,
}
}
pub fn target_kbps(&self) -> u32 {
self.target_kbps
}
pub fn limits(&self) -> Limits {
self.limits
}
pub fn mode(&self) -> ControlMode {
self.mode
}
pub fn reason(&self) -> Reason {
self.reason
}
/// How far behind the send queue was at the last decision, in milliseconds.
pub fn backlog_ms(&self) -> u32 {
self.backlog_ms
}
/// The ceiling the box was given, whatever a client has since asked for.
pub fn box_ceiling_kbps(&self) -> u32 {
self.box_ceiling_kbps
}
/// Adopt a ceiling a client asked for, never above the one the box was given.
///
/// A client may lower its own ceiling -- to test a path, or because it knows
/// something about its link that this end does not -- but it may not raise
/// the one the tier bought.
pub fn set_ceiling(&mut self, ceiling_kbps: u32) {
self.limits = Limits::new(ceiling_kbps.min(self.box_ceiling_kbps).max(1));
self.target_kbps = self.limits.clamp(self.target_kbps);
// A ceiling that moved is worth restating even when the target did not,
// because the encoder is the thing that has to hear about it.
self.sent_kbps = None;
}
pub fn set_mode(&mut self, mode: ControlMode) {
self.mode = mode;
if mode == ControlMode::Manual {
self.reason = Reason::Manual;
}
}
/// Note that somebody set the bitrate by hand.
pub fn note_manual_target(&mut self, kbps: u32) {
self.target_kbps = kbps;
self.sent_kbps = Some(kbps);
self.mode = ControlMode::Manual;
self.reason = Reason::Manual;
}
/// Note that the encoder is, or is no longer, under constant quality.
pub fn set_constant_quality(&mut self, constant_quality: bool) {
self.constant_quality = constant_quality;
}
/// One second's decision.
///
/// `report` is the worst report across the clients attached, or `None` when
/// none of them said anything. Returns the new target when it is worth
/// sending, and `None` when nothing should be sent -- which is most seconds.
/// One second's decision.
///
/// `self_inflicted` says this second contained frames the hub deliberately
/// withheld -- a client resynchronising, whose deltas were undecodable and
/// were dropped rather than sent. Those seconds cannot be read as evidence
/// about the path: fewer frames were sent, so fewer arrived, so the measured
/// goodput is low and the loss is high, and a controller anchoring a backoff
/// on that would cut the bitrate in response to its own decision. Every
/// stutter would then also cost bandwidth, which is the opposite of what a
/// resynchronisation needs.
pub fn tick(
&mut self,
clients: usize,
report: Option<ReceiverReport>,
path: PathView,
self_inflicted: bool,
) -> Option<u32> {
if clients == 0 {
// Nothing is connected, so nothing is being carried and there is no
// path to form an opinion about. Deciding here means deciding on the
// absence of evidence: the controller used to read it as silence and
// decay to the floor, so a box waiting for its first client spent
// that time winding itself down and then jumped back up the moment
// somebody arrived.
self.reason = Reason::NoClients;
self.silent_ticks = 0;
return None;
}
if self.constant_quality {
self.reason = Reason::ConstantQuality;
return None;
}
if self.mode == ControlMode::Manual {
self.reason = Reason::Manual;
return None;
}
if self_inflicted {
// Held, not decayed. The path may be fine; this second simply
// cannot say, and silence about the path is not evidence against it.
self.reason = Reason::SelfInflicted;
self.silent_ticks = 0;
return None;
}
// What the queue drains at is what is getting through. With no report
// to say, the target is the best guess available -- and a target that
// is too high only makes the backlog look shorter than it is, so this
// errs towards patience rather than towards cutting the rate.
let drain_kbps = match report.as_ref().map(|r| (r.goodput_bps / 1000) as u32) {
Some(measured) if measured > 0 => measured,
_ => self.target_kbps,
};
self.backlog_ms = path.backlog_ms(drain_kbps).unwrap_or(0);
let next = match report.as_ref().and_then(|r| r.loss().map(|l| (r, l))) {
Some((report, loss)) => {
self.silent_ticks = 0;
self.decide_from_report(report, loss, path)
}
None => {
self.silent_ticks = self.silent_ticks.saturating_add(1);
if self.silent_ticks < SILENT_TICKS_BEFORE_FALLBACK {
self.reason = Reason::Holding;
return None;
}
self.decide_from_path(path)
}
};
self.target_kbps = self.limits.clamp(next);
// Every second moves the belief; only a change worth acting on is said.
// The first decision is always said -- the encoder started at whatever
// its environment gave it, and this end has no way to know it matches.
match self.sent_kbps {
Some(sent) if !worth_sending(sent, self.target_kbps, self.limits) => None,
_ => {
self.sent_kbps = Some(self.target_kbps);
Some(self.target_kbps)
}
}
}
fn decide_from_report(&mut self, report: &ReceiverReport, loss: f32, path: PathView) -> u32 {
let measured = (report.goodput_bps / 1000) as u32;
// `backlog_ms` is zero both when the queue is empty and when the
// transport cannot say, which are the same thing for this purpose: a
// queue nobody can see is not evidence of one.
let known = path.backlog_bytes.is_some();
let queued = known && self.backlog_ms > QUEUE_DECREASE_MS;
if loss > LOSS_DECREASE || queued {
self.reason = if queued {
Reason::Backlogged
} else {
Reason::Congested
};
// Anchored on what actually arrived, not on what we were asking for.
// `goodput` of zero means nothing completed at all, in which case
// there is no measurement to anchor to and the target is simply
// halved -- the alternative, backing off to zero, would take the
// stream below the floor on a single bad second.
let anchor = if measured == 0 {
self.target_kbps / 2
} else {
measured.min(self.target_kbps)
};
return (anchor as f32 * BACKOFF) as u32;
}
// Climbing is what digs the queue, so it needs the queue to be empty as
// well as the loss to be low. Without this the controller climbs all
// the way to the ceiling against a path it is already overrunning,
// because a path that queues instead of dropping reports no loss at all
// until the buffer finally overflows -- and by then the picture is
// seconds behind.
if loss < LOSS_INCREASE && (!known || self.backlog_ms < QUEUE_CLIMB_MS) {
self.reason = Reason::Climbing;
return self
.target_kbps
.saturating_add(self.limits.ceiling_kbps / CLIMB_FRACTION);
}
self.reason = Reason::Holding;
self.target_kbps
}
fn decide_from_path(&mut self, path: PathView) -> u32 {
match path.estimate_kbps() {
Some(estimate) => {
self.reason = Reason::Fallback;
estimate
}
None => {
// Nothing said and nothing visible. Holding a high target on no
// evidence is how the original failure sustained itself, so this
// decays rather than holds.
self.reason = Reason::Blind;
(self.target_kbps as f32 * BACKOFF) as u32
}
}
}
}
/// Whether a change is big enough to be worth actuating.
///
/// Reaching either end of the band always is: a target pinned to the floor or
/// the ceiling is a fact worth stating even when the step to it was small.
pub fn worth_sending(current: u32, next: u32, limits: Limits) -> bool {
if current == next {
return false;
}
if next == limits.floor_kbps() || next == limits.ceiling_kbps {
return true;
}
let change = current.abs_diff(next) as f32;
change >= current as f32 * DEADBAND
}
#[cfg(test)]
mod tests {
use super::*;
const CEILING: u32 = 10_000;
fn limits() -> Limits {
Limits::new(CEILING)
}
fn controller() -> Controller {
Controller::new(limits())
}
/// A second in which `released` of `total` frames made it, carrying
/// `goodput_kbps` of completed video.
fn report(released: u32, incomplete: u32, goodput_kbps: u64) -> ReceiverReport {
ReceiverReport {
goodput_bps: goodput_kbps * 1000,
released,
incomplete,
never_arrived: 0,
rtt_ms: 180,
}
}
fn healthy() -> ReceiverReport {
report(60, 0, 9_800)
}
/// A path with `backlog` bytes handed over and not yet gone.
fn backlogged(backlog: u64) -> PathView {
PathView {
cwnd_bytes: Some(13_000),
rtt_ms: Some(180),
backlog_bytes: Some(backlog),
}
}
/// The second reported failure, as measured over a 1000-mile link.
///
/// Every frame arrived and every frame completed -- zero loss, zero QUIC
/// congestion events, a flat 180 ms round trip for the whole session -- and
/// the picture was still several seconds behind, because the offer was
/// several times what the path carried and the difference was sitting in
/// the hub's own send buffer. A controller reading loss alone sees a
/// perfect path here and climbs to the ceiling against it.
#[test]
fn a_path_that_queues_instead_of_dropping_must_not_read_as_healthy() {
let mut c = controller();
// Two megabits get through; a megabyte is already queued behind them.
let arriving = report(60, 0, 2_000);
let path = backlogged(1024 * 1024);
for _ in 0..30 {
c.tick(1, Some(arriving), path, false);
}
assert_ne!(
c.reason(),
Reason::Climbing,
"climbing against a path with four seconds of backlog"
);
assert!(
c.target_kbps() < 2_000,
"target {} kbps is at or above what is getting through, so the \
queue can only grow",
c.target_kbps()
);
}
/// The sawtooth: floor, climb to ceiling, collapse, repeat every 20 s.
///
/// Loss alone cannot break this cycle, because on a queueing path loss only
/// appears once the buffer finally overflows -- long after the latency has
/// made the session unplayable, and by then the queue is deep enough that
/// backing off to the floor is the only way out.
#[test]
fn the_target_settles_instead_of_sawtoothing() {
let mut c = controller();
// A steady 2 Mbps path. The backlog is what the last second of
// over-sending left behind, drained at what actually gets through.
let mut backlog: i64 = 0;
let mut seen = Vec::new();
for _ in 0..60 {
let target = c.target_kbps();
// Whatever was asked for above 2 Mbps piles up; the rest drains.
backlog = (backlog + (i64::from(target) - 2_000) * 1000 / 8).clamp(0, 4 * 1024 * 1024);
c.tick(
1,
Some(report(60, 0, 2_000)),
backlogged(backlog as u64),
false,
);
seen.push(c.target_kbps());
}
let settled = &seen[30..];
let (lo, hi) = (
*settled.iter().min().unwrap(),
*settled.iter().max().unwrap(),
);
assert!(
hi - lo <= 1_000,
"target still swinging between {lo} and {hi} kbps after 30 seconds"
);
assert!(
hi <= 2_400,
"settled at {hi} kbps against a path carrying 2000"
);
}
/// The reported depth is what the next session will be judged on, so it
/// has to be right: bytes over the rate they leave at, in milliseconds.
#[test]
fn the_queue_depth_is_reported_as_measured() {
let mut c = controller();
// 250 kB queued behind a 2 Mbps drain is exactly one second.
c.tick(1, Some(report(60, 0, 2_000)), backlogged(250_000), false);
assert_eq!(c.backlog_ms(), 1_000);
// A transport that cannot say must report no queue, not a wrong one.
c.tick(1, Some(report(60, 0, 2_000)), PathView::default(), false);
assert_eq!(c.backlog_ms(), 0);
}
/// The backlog must not become a reason never to climb again.
#[test]
fn a_clear_queue_still_climbs() {
let mut c = controller();
c.set_ceiling(4_000);
let clear = backlogged(0);
for _ in 0..5 {
c.tick(1, Some(report(60, 0, 3_900)), clear, false);
}
assert_eq!(c.reason(), Reason::Climbing);
}
/// The reported failure, as measured: a 10 Mbps offer into a path carrying
/// under three, where *no* frame completed for minutes.
#[test]
fn the_reported_failure_is_corrected_in_seconds() {
let mut c = controller();
assert_eq!(c.target_kbps(), CEILING);
let mut ticks = 0;
while c.target_kbps() > 2_800 && ticks < 10 {
// 46 frames a second starting to arrive, none of them completing.
c.tick(1, Some(report(0, 46, 0)), PathView::default(), false);
ticks += 1;
}
assert!(
c.target_kbps() <= 2_800,
"still at {} kbps after {ticks} seconds",
c.target_kbps(),
);
assert!(
ticks <= 5,
"took {ticks} seconds to stop overrunning the path"
);
}
/// With a goodput measurement to anchor on it should take one step, not
/// several -- that is the whole reason for backing off to what arrived
/// rather than to a fraction of what was asked for.
#[test]
fn a_measured_goodput_is_corrected_in_one_step() {
let mut c = controller();
let sent = c.tick(1, Some(report(12, 48, 2_850)), PathView::default(), false);
assert_eq!(sent, Some(c.target_kbps()));
assert!(
c.target_kbps() < 2_850,
"backed off to {} kbps, which is above what actually arrived",
c.target_kbps(),
);
}
/// The negative that matters most: a path that is fine must be left alone.
#[test]
fn a_healthy_path_is_not_wandered_away_from() {
let mut c = controller();
// The first decision is always stated. The encoder started at whatever
// its environment gave it and this end cannot know that matches, so the
// target is asserted once rather than assumed.
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(CEILING)
);
for _ in 0..60 {
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
None,
"a healthy path produced a bitrate change",
);
}
assert_eq!(c.target_kbps(), CEILING);
}
#[test]
fn the_first_decision_is_always_stated() {
// Otherwise a session whose path happens to match its ceiling never
// tells the encoder anything, and the encoder keeps whatever its
// environment gave it -- which is the failure this replaces.
let mut c = controller();
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(CEILING),
"the opening target was never stated",
);
// And not repeated, now that the encoder has been told.
assert_eq!(c.tick(1, Some(healthy()), PathView::default(), false), None);
}
#[test]
fn a_small_change_is_not_worth_saying() {
// Below the deadband nothing is sent, so the log and the encoder are not
// touched sixty times a minute for changes nobody could see.
let l = limits();
assert!(!worth_sending(5_000, 5_200, l));
assert!(worth_sending(5_000, 5_600, l));
}
#[test]
fn reaching_either_end_of_the_band_is_always_worth_saying() {
let l = limits();
assert!(worth_sending(l.floor_kbps() + 1, l.floor_kbps(), l));
assert!(worth_sending(CEILING - 1, CEILING, l));
}
#[test]
fn the_ceiling_is_never_exceeded() {
let mut c = controller();
for _ in 0..200 {
c.tick(1, Some(healthy()), PathView::default(), false);
assert!(c.target_kbps() <= CEILING, "{} kbps", c.target_kbps());
}
}
#[test]
fn the_floor_is_never_gone_below() {
let mut c = controller();
for _ in 0..200 {
c.tick(1, Some(report(0, 60, 0)), PathView::default(), false);
assert!(
c.target_kbps() >= c.limits().floor_kbps(),
"{} kbps is below the floor",
c.target_kbps(),
);
}
}
#[test]
fn it_climbs_back_after_a_bad_patch() {
let mut c = controller();
for _ in 0..5 {
c.tick(1, Some(report(0, 46, 0)), PathView::default(), false);
}
let bottom = c.target_kbps();
for _ in 0..40 {
c.tick(1, Some(healthy()), PathView::default(), false);
}
assert!(
c.target_kbps() > bottom,
"never recovered from {bottom} kbps"
);
assert_eq!(c.target_kbps(), CEILING, "did not climb all the way back");
}
#[test]
fn a_second_the_hub_starved_is_not_read_as_a_bad_path() {
// While a client resynchronises its deltas are withheld, so fewer frames
// are sent, fewer arrive, goodput reads low and loss reads high. Backing
// off on that would cut the bitrate in response to the hub's own
// decision -- and make every stutter cost bandwidth as well.
let mut c = controller();
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(CEILING)
);
let starved = report(0, 46, 0);
for _ in 0..10 {
assert_eq!(c.tick(1, Some(starved), PathView::default(), true), None);
}
assert_eq!(c.target_kbps(), CEILING, "the hub cut its own bitrate");
assert_eq!(c.reason(), Reason::SelfInflicted);
}
#[test]
fn a_genuinely_bad_second_still_acts_once_the_hub_stops_starving_it() {
// The flag holds, it does not blind. The moment a second is the path's
// own, the same evidence is acted on.
let mut c = controller();
c.tick(1, Some(healthy()), PathView::default(), false);
c.tick(1, Some(report(0, 46, 0)), PathView::default(), true);
assert_eq!(c.target_kbps(), CEILING);
c.tick(1, Some(report(12, 48, 2_850)), PathView::default(), false);
assert!(c.target_kbps() < 2_850, "a real bad second was ignored too");
}
#[test]
fn a_starved_second_does_not_count_towards_silence() {
// It is not a missing report -- one arrived, it just cannot be read.
// Counting it as silence would slide towards the path-based fallback and
// then to decaying blind, which is a different wrong answer.
let mut c = controller();
c.tick(1, Some(healthy()), PathView::default(), false);
for _ in 0..10 {
c.tick(1, Some(report(0, 60, 0)), PathView::default(), true);
}
assert_eq!(c.reason(), Reason::SelfInflicted);
assert_eq!(c.target_kbps(), CEILING);
}
#[test]
fn a_hand_set_bitrate_stands_the_controller_down() {
// How this class of bug gets diagnosed. A controller that overrode a
// person's setting a second later would take the tool away.
let mut c = controller();
c.note_manual_target(1_000);
for _ in 0..30 {
assert_eq!(
c.tick(1, Some(report(0, 60, 0)), PathView::default(), false),
None
);
}
assert_eq!(c.target_kbps(), 1_000);
assert_eq!(c.reason(), Reason::Manual);
}
#[test]
fn constant_quality_has_no_bitrate_to_decide() {
let mut c = controller();
c.set_constant_quality(true);
assert_eq!(
c.tick(1, Some(report(0, 60, 0)), PathView::default(), false),
None
);
assert_eq!(c.reason(), Reason::ConstantQuality);
}
#[test]
fn silence_is_tolerated_briefly_and_then_acted_on() {
let mut c = controller();
// A missed report or two says nothing; the path was fine a second ago.
for _ in 0..(SILENT_TICKS_BEFORE_FALLBACK - 1) {
assert_eq!(c.tick(1, None, PathView::default(), false), None);
assert_eq!(c.target_kbps(), CEILING);
}
// Past that, with nothing visible from this end either, it decays rather
// than holding a high target on no evidence at all -- holding is how the
// original failure sustained itself.
c.tick(1, None, PathView::default(), false);
assert_eq!(c.reason(), Reason::Blind);
assert!(c.target_kbps() < CEILING);
}
#[test]
fn this_ends_view_is_used_only_when_the_far_end_is_silent() {
let mut c = controller();
let path = PathView {
cwnd_bytes: Some(120_000),
rtt_ms: Some(200),
backlog_bytes: None,
};
// A report present means the path view is ignored, however tempting.
c.tick(1, Some(healthy()), path, false);
assert_eq!(c.reason(), Reason::Climbing);
for _ in 0..SILENT_TICKS_BEFORE_FALLBACK {
c.tick(1, None, path, false);
}
assert_eq!(c.reason(), Reason::Fallback);
// 120 KB in flight per 200 ms is 4.8 Mbps.
assert_eq!(c.target_kbps(), 4_800);
}
#[test]
fn nothing_is_decided_while_nobody_is_connected() {
// A box waiting for its first client used to read the silence as a dead
// path and wind itself down to the floor, then jump back up the moment
// somebody arrived. There is no path to have an opinion about yet.
let mut c = controller();
for _ in 0..30 {
assert_eq!(c.tick(0, None, PathView::default(), false), None);
}
assert_eq!(c.target_kbps(), CEILING);
assert_eq!(c.reason(), Reason::NoClients);
}
#[test]
fn a_lowered_ceiling_can_be_raised_again() {
// The trap in folding the box's ceiling together with the one in force:
// after lowering, the only number left to compare against is the lowered
// one, so the client can never get back up.
let mut c = controller();
c.set_ceiling(2_000);
assert_eq!(c.limits().ceiling_kbps, 2_000);
c.set_ceiling(5_000);
assert_eq!(c.limits().ceiling_kbps, 5_000);
assert_eq!(c.box_ceiling_kbps(), CEILING);
}
#[test]
fn a_ceiling_change_is_restated_to_the_encoder() {
// The encoder is the thing that has to act on it, and a target that
// happens to land on the same number is still a different instruction
// when the band around it moved.
let mut c = controller();
assert!(
c.tick(1, Some(healthy()), PathView::default(), false)
.is_some()
);
c.set_ceiling(3_000);
assert_eq!(
c.tick(1, Some(healthy()), PathView::default(), false),
Some(3_000)
);
}
#[test]
fn a_client_may_lower_its_ceiling_but_not_raise_it() {
let mut c = controller();
c.set_ceiling(4_000);
assert_eq!(c.limits().ceiling_kbps, 4_000);
assert!(c.target_kbps() <= 4_000, "the target outlived its ceiling");
c.set_ceiling(50_000);
assert_eq!(
c.limits().ceiling_kbps,
CEILING,
"a client raised the ceiling the tier bought",
);
}
#[test]
fn a_ceiling_under_the_floor_does_not_invert_the_band() {
// A misconfigured tier must not produce a floor above its own ceiling,
// which would put every target above the limit it exists to enforce.
let l = Limits::new(500);
assert!(l.floor_kbps() <= l.ceiling_kbps);
assert_eq!(l.clamp(10_000), 500);
}
#[test]
fn a_silent_second_is_not_read_as_a_healthy_one() {
// No frames at all accounted for: nothing sent, or nothing arrived.
// Reading it as no loss would climb straight into a path that may be
// carrying nothing.
let mut c = controller();
let empty = ReceiverReport::default();
assert_eq!(empty.loss(), None);
c.tick(1, Some(empty), PathView::default(), false);
assert_eq!(c.target_kbps(), CEILING, "climbed on an empty report");
}
}
apps/neshub/src/dgram.rs
+204 -31
View File
@@ -143,6 +143,73 @@ impl DatagramSender {
}
}
/// How long deltas may be withheld from a client waiting to resynchronise.
///
/// A bound rather than a belief. Withholding is correct only while the keyframe
/// is actually coming, and if the encoder never produces one -- it refused, it
/// died, the request never reached it -- then withholding forever turns a
/// recoverable freeze into a permanent black screen. Past this the deltas go out
/// again: useless to a desynchronised decoder, but "useless" beats "nothing at
/// all, forever" when the assumption behind the suppression has been proven
/// wrong.
const MAX_RESYNC_WITHHOLD: std::time::Duration = std::time::Duration::from_secs(2);
/// Whether to send delta frames to a client that cannot decode them yet.
///
/// A client that has lost synchronisation asks for a keyframe, and until one
/// arrives every delta frame sent to it is undecodable -- it predicts from
/// pictures that client does not have. Those frames are not merely wasted: quinn
/// writes DATAGRAM frames into a packet before STREAM frames, so a steady stream
/// of deltas takes the space the keyframe needs and starves the one frame that
/// would end the freeze. That is the loop behind "26 keyframe fallbacks, 19 IDR
/// requests": the recovery frame could not get out past the frames that needed
/// it to arrive first.
///
/// So while a client is waiting, its deltas are dropped rather than sent.
#[derive(Debug, Default)]
pub struct ResyncGate {
/// When this client started waiting, or `None` if it is not.
waiting_since: Option<std::time::Instant>,
/// Deltas dropped during the current wait.
withheld: u64,
/// Set once the withhold bound is passed, so it is said once per wait.
gave_up: bool,
}
impl ResyncGate {
/// Whether this delta frame should go out.
///
/// `awaiting` is whether the client has asked for a keyframe and not yet
/// been sent one.
pub fn admit_delta(&mut self, awaiting: bool, now: std::time::Instant) -> bool {
if !awaiting {
self.end_wait();
return true;
}
let since = *self.waiting_since.get_or_insert(now);
if now.duration_since(since) >= MAX_RESYNC_WITHHOLD {
self.gave_up = true;
return true;
}
self.withheld += 1;
false
}
/// A keyframe has gone out, so the wait is over.
pub fn note_keyframe(&mut self) -> Option<(u64, bool)> {
let withheld = self.withheld;
let gave_up = self.gave_up;
self.end_wait();
(withheld > 0).then_some((withheld, gave_up))
}
fn end_wait(&mut self) {
self.waiting_since = None;
self.withheld = 0;
self.gave_up = false;
}
}
/// Frames arriving on `rx` are numbered and sent — deltas as datagrams,
/// keyframes on a reliable stream each when `keyframes_reliable` is set.
///
@@ -161,6 +228,12 @@ pub async fn run_datagram_writer(
mut rx: tokio::sync::mpsc::UnboundedReceiver<Vec<u8>>,
relay_ms: Option<Arc<AtomicU32>>,
keyframes_reliable: bool,
// Set while this client has asked for a keyframe and not yet been sent
// one. Video only; audio has no such notion.
awaiting_keyframe: Option<Arc<std::sync::atomic::AtomicBool>>,
// Counts frames withheld during a resynchronisation, so the bitrate
// controller can tell a second it starved from a second the path did.
withheld: Option<Arc<std::sync::atomic::AtomicU64>>,
) {
let sender = DatagramSender::new(conn.clone(), kind);
let keyframes = keyframes_reliable.then(|| KeyframeSender::new(conn, sender.clone()));
@@ -170,33 +243,54 @@ pub async fn run_datagram_writer(
// debug. Losing datagram support entirely is not, and is worth a warning —
// but only the first time, since it will then be true for every frame.
let mut warned_unsupported = false;
// Where this writer's second went.
//
// The pair that matters is the first two: if frames arrive here already
// 43 ms apart then the hole was made upstream, in the encoder or on the IPC
// hop, and nothing in this file can be the cause. If they arrive evenly and
// leave unevenly, it is made here. A client measured exactly that hole in
// video datagram arrivals while audio — same connection, same congestion
// window, its own writer — stayed at 8 ms.
let mut last_in = std::time::Instant::now();
let mut worst_in_gap = std::time::Duration::ZERO;
let mut worst_send = std::time::Duration::ZERO;
let mut frames: u32 = 0;
let mut last_pace = std::time::Instant::now();
let mut resync = ResyncGate::default();
while let Some(payload) = rx.recv().await {
let t0 = std::time::Instant::now();
worst_in_gap = worst_in_gap.max(t0.duration_since(last_in));
last_in = t0;
frames += 1;
body.clear();
nesprotocol::encode_frame_body(&mut body, MSG_DATA, seq, &payload);
// A keyframe goes on a stream of its own when one will take it. The
// relay timing below is not recorded for it: the write is asynchronous
// by design, so the time this loop spent on it says nothing.
// A client that cannot decode has asked for a keyframe; until it gets
// one, everything else sent to it is undecodable and takes the space the
// keyframe needs. See `ResyncGate`.
if let Some(ref awaiting) = awaiting_keyframe {
if nesprotocol::reliable::video_is_keyframe(&payload) {
awaiting.store(false, Ordering::Relaxed);
if let Some((withheld, gave_up)) = resync.note_keyframe() {
debug!(
"{label}: resynchronised after withholding {withheld} frame(s){}",
if gave_up {
", having given up waiting"
} else {
""
}
);
}
} else if !resync
.admit_delta(awaiting.load(Ordering::Relaxed), std::time::Instant::now())
{
// The sequence number deliberately does *not* advance. A
// withheld frame was never sent, so leaving a hole would make
// the receiver count it as lost -- and that count is what the
// bitrate controller reads. The hub would then lower the
// bitrate because of frames it chose not to send, which is a
// controller reacting to its own decision rather than to the
// path. Reusing the number is safe precisely because nothing
// went out under it.
//
// Saying so is the other half: the receiver still reports fewer
// frames this second, and without this count the controller has
// no way to tell that second from one the path ruined.
if let Some(ref withheld) = withheld {
withheld.fetch_add(1, Ordering::Relaxed);
}
continue;
}
}
if let Some(ref keyframes) = keyframes
&& video_wants_reliable(&payload)
{
@@ -229,19 +323,6 @@ pub async fn run_datagram_writer(
}
Err(e) => debug!("{label}: dropping frame {seq}: {e}"),
}
worst_send = worst_send.max(t0.elapsed());
if last_pace.elapsed() >= std::time::Duration::from_secs(1) {
last_pace = std::time::Instant::now();
debug!(
"{label}: {frames} frames, worst gap between frames in {:.1}ms, worst send {:.1}ms",
worst_in_gap.as_secs_f64() * 1000.0,
worst_send.as_secs_f64() * 1000.0,
);
worst_in_gap = std::time::Duration::ZERO;
worst_send = std::time::Duration::ZERO;
frames = 0;
}
seq = seq.wrapping_add(1);
}
@@ -252,3 +333,95 @@ pub async fn run_datagram_writer(
}
debug!("{label} datagram writer exiting (channel closed)");
}
#[cfg(test)]
mod resync_gate_tests {
use super::{MAX_RESYNC_WITHHOLD, ResyncGate};
use std::time::{Duration, Instant};
#[test]
fn a_client_that_can_decode_gets_everything() {
// The negative that matters: nothing is withheld from a healthy client.
let mut gate = ResyncGate::default();
let now = Instant::now();
for i in 0..1000 {
assert!(gate.admit_delta(false, now + Duration::from_millis(i)));
}
}
#[test]
fn a_waiting_client_is_not_sent_frames_it_cannot_decode() {
let mut gate = ResyncGate::default();
let now = Instant::now();
for i in 0..60 {
assert!(
!gate.admit_delta(true, now + Duration::from_millis(i * 16)),
"frame {i} went to a client with no reference to decode it against",
);
}
}
#[test]
fn the_keyframe_ends_the_wait_and_reports_the_cost() {
let mut gate = ResyncGate::default();
let now = Instant::now();
for i in 0..5 {
gate.admit_delta(true, now + Duration::from_millis(i * 16));
}
assert_eq!(gate.note_keyframe(), Some((5, false)));
// And the next delta goes out immediately.
assert!(gate.admit_delta(false, now + Duration::from_millis(100)));
}
#[test]
fn a_keyframe_with_nothing_withheld_says_nothing() {
// So an ordinary periodic keyframe does not log a recovery that did not
// happen.
let mut gate = ResyncGate::default();
assert_eq!(gate.note_keyframe(), None);
}
#[test]
fn withholding_gives_up_rather_than_going_dark_forever() {
// The bound. Withholding is only correct while the keyframe is actually
// coming; if the encoder never produces one, suppressing forever turns a
// recoverable freeze into a permanent black screen. Undecodable frames
// beat no frames once the assumption is disproven.
let mut gate = ResyncGate::default();
let now = Instant::now();
assert!(!gate.admit_delta(true, now));
assert!(!gate.admit_delta(true, now + MAX_RESYNC_WITHHOLD - Duration::from_millis(1)));
assert!(
gate.admit_delta(true, now + MAX_RESYNC_WITHHOLD),
"still withholding after the keyframe plainly is not coming",
);
}
#[test]
fn giving_up_is_reported_when_the_keyframe_finally_lands() {
let mut gate = ResyncGate::default();
let now = Instant::now();
gate.admit_delta(true, now);
gate.admit_delta(true, now + MAX_RESYNC_WITHHOLD);
let (withheld, gave_up) = gate.note_keyframe().expect("something was withheld");
assert_eq!(withheld, 1);
assert!(gave_up, "the wait timed out and nothing said so");
}
#[test]
fn a_second_wait_starts_fresh() {
// Otherwise the first wait's elapsed time carries over and the second is
// abandoned immediately, or its count reports the wrong recovery.
let mut gate = ResyncGate::default();
let now = Instant::now();
gate.admit_delta(true, now);
gate.note_keyframe();
let later = now + Duration::from_secs(60);
assert!(
!gate.admit_delta(true, later),
"the new wait was not honoured"
);
assert_eq!(gate.note_keyframe(), Some((1, false)));
}
}
apps/neshub/src/main.rs
+155 -12
View File
@@ -1,3 +1,4 @@
mod control;
mod dgram;
mod ipc_listener;
mod keyframe;
@@ -14,7 +15,7 @@ use iroh::endpoint::presets;
use crate::session::SessionManager;
use crate::ticket::NestriTicket;
use nesprotocol::ALPN;
use nesprotocol::{ALPNS, Carrier};
#[derive(Parser, Debug)]
#[command(name = "neshub")]
@@ -73,6 +74,16 @@ struct Args {
#[arg(long, env = "NESTRI_AUDIO_BITRATE", default_value_t = 64)]
audio_bitrate_per_channel: u32,
/// Ceiling on the video bitrate, in kbps.
///
/// Set by `nesinit` from the boot descriptor's video limits, which come
/// from the tier the box was sized for. Absent means nobody said -- which is
/// not a licence to send whatever the encoder defaults to, since that is
/// precisely how every session came to offer 10 Mbps regardless of what the
/// path could carry. Unset is reported, and a conservative ceiling is used.
#[arg(long, env = "NESTRI_MAX_BITRATE")]
max_bitrate_kbps: Option<u32>,
/// Socket nescope sends screenshots on. neshub listens; nescope dials out.
#[arg(
long,
@@ -82,6 +93,14 @@ struct Args {
screenshot_ipc: PathBuf,
}
/// What to assume when nobody said.
///
/// Deliberately modest. A ceiling that was never set should not behave like an
/// unlimited one: the whole failure this exists to fix was a session offering
/// 10 Mbps into a path carrying under three, because no number had ever been
/// chosen and the encoder's own default stood in for one.
const DEFAULT_MAX_BITRATE_KBPS: u32 = 4_000;
#[tokio::main]
async fn main() -> Result<()> {
tracing_subscriber::fmt()
@@ -93,7 +112,7 @@ async fn main() -> Result<()> {
let args = Args::parse();
let mut builder = iroh::Endpoint::builder(presets::N0)
.alpns(vec![ALPN.to_vec()])
.alpns(ALPNS.iter().map(|a| a.to_vec()).collect::<Vec<_>>())
.transport_config(crate::dgram::media_transport_config());
match args.relay.as_str() {
@@ -117,6 +136,15 @@ async fn main() -> Result<()> {
}
}
match args.max_bitrate_kbps {
Some(kbps) => tracing::info!("video ceiling: {kbps} kbps, from the boot descriptor"),
None => tracing::warn!(
"no video ceiling on the boot descriptor; using {DEFAULT_MAX_BITRATE_KBPS} kbps. \
A box sized by a tier is told its ceiling -- if this is one, the descriptor did \
not carry it."
),
}
let endpoint = builder.bind().await?;
let endpoint_addr = endpoint.addr();
let ep_id = endpoint_addr.id;
@@ -134,6 +162,14 @@ async fn main() -> Result<()> {
let session_manager = Arc::new(SessionManager::new());
// One controller for the box, not one per client: there is one encoder, so
// there is one bitrate, and the client having the worst time is the one it
// has to answer.
let box_ceiling_kbps = args.max_bitrate_kbps.unwrap_or(DEFAULT_MAX_BITRATE_KBPS);
let controller = Arc::new(tokio::sync::Mutex::new(control::Controller::new(
control::Limits::new(box_ceiling_kbps),
)));
// IDR / encode settings command channel: input reader → nescapture
let (cmd_tx, mut cmd_rx) = tokio::sync::mpsc::unbounded_channel::<Vec<u8>>();
{
@@ -181,23 +217,105 @@ async fn main() -> Result<()> {
args.audio_channels,
args.audio_bitrate_per_channel
);
let controller = controller.clone();
let cmd_tx = cmd_tx.clone();
tokio::spawn(async move {
// Ten times a second, and only when asked for. Pairs with
// nescapture's rate probe: that measures how fast the encoder
// follows a new bitrate, this shows how fast the queue downstream
// of it responds, and the slower of the two is the fastest a
// control loop can usefully run. At one sample a second neither is
// visible -- a queue that fills and drains inside a second looks
// like a queue that was never there.
if std::env::var("NESHUB_BACKLOG_TRACE").is_ok_and(|v| v != "0" && !v.is_empty()) {
let mgr = mgr.clone();
let controller = controller.clone();
tokio::spawn(async move {
let mut fast = tokio::time::interval(std::time::Duration::from_millis(100));
tracing::info!("backlog trace on, 10 Hz");
loop {
fast.tick().await;
let backlogs = mgr.backlog_bytes().await;
if backlogs.is_empty() {
continue;
}
let target = controller.lock().await.target_kbps();
for bytes in backlogs {
// Against the target rather than a measured drain:
// this is a trace to read afterwards, and a number
// divided by a second measurement is two things
// moving at once.
let ms = bytes.saturating_mul(8) / u64::from(target.max(1));
tracing::info!("backlog {ms} ms ({bytes} bytes) at {target} kbps");
}
}
});
}
let mut interval = tokio::time::interval(std::time::Duration::from_secs(1));
loop {
interval.tick().await;
// One decision a second, on the same tick as the stats, because
// a report describes the second that just passed and there is
// nothing to gain from deciding more often than they arrive.
{
let clients = mgr.client_count().await;
let (report, path, self_inflicted) = mgr.worst_report().await;
let mut controller = controller.lock().await;
if let Some(kbps) = controller.tick(clients, report, path, self_inflicted) {
let mut cmd = vec![nesprotocol::MSG_ENCODE_SETTINGS];
nesprotocol::encode_bitrate_only(&mut cmd, kbps);
if cmd_tx.send(cmd).is_err() {
tracing::warn!("encoder command channel closed");
} else {
// Trace, not info: under a path that keeps moving
// this fires every second, and a per-second line at
// info buries everything worth reading.
tracing::trace!(
"video ceiling {}/{} kbps: {:?}",
kbps,
controller.limits().ceiling_kbps,
controller.reason(),
);
}
}
}
let clients = mgr.client_count().await as u8;
let bitrate = mgr.video_bitrate_bps();
let (key_bps, delta_bps, keyframes) = mgr.video_breakdown();
let (pipeline_p50_ms, pipeline_p95_ms, pipeline_max_ms) = mgr.pipeline_delays();
let audio_kbps = mgr.audio_bitrate_kbps();
let relay_ms = mgr.relay_ms();
let mut buf = Vec::with_capacity(15);
let mut buf = Vec::with_capacity(34);
nesprotocol::stats::encode_hub_stats(
&mut buf,
clients,
bitrate,
key_bps.saturating_add(delta_bps),
relay_ms,
audio_kbps,
audio_channels,
);
{
let controller = controller.lock().await;
nesprotocol::stats::encode_video_breakdown(
&mut buf,
&nesprotocol::stats::VideoBreakdown {
key_bps,
delta_bps,
keyframes,
target_kbps: controller.target_kbps(),
ceiling_kbps: controller.limits().ceiling_kbps,
reason: controller.reason() as u8,
manual: u8::from(controller.mode() == nesprotocol::ControlMode::Manual),
box_ceiling_kbps: controller.box_ceiling_kbps(),
pipeline_p50_ms,
pipeline_p95_ms,
pipeline_max_ms,
backlog_ms: controller.backlog_ms().min(u32::from(u16::MAX)) as u16,
},
);
}
mgr.broadcast_stats(buf).await;
}
});
@@ -262,18 +380,43 @@ async fn main() -> Result<()> {
match incoming.await {
Ok(conn) => {
let remote_id = conn.remote_id();
tracing::info!(remote = %remote_id.fmt_short(), "client connected");
let session = session::ClientSession::new(
// A client opens one connection per kind of traffic, so the
// ALPN says which this is and the endpoint id says whose.
let Some(carrier) = Carrier::from_alpn(conn.alpn()) else {
tracing::warn!(
remote = %remote_id.fmt_short(),
"connection with an unknown ALPN; closing"
);
conn.close(0u32.into(), b"unknown alpn");
continue;
};
tracing::info!(
remote = %remote_id.fmt_short(),
carrier = carrier.label(),
"client connected"
);
mgr.attach(
remote_id,
carrier,
conn.clone(),
input_broadcast_tx.clone(),
session_manager.relay_ms_atomic(),
cmd_tx.clone(),
);
mgr.add_session(remote_id, session).await;
controller.clone(),
)
.await;
let mgr_clone = mgr.clone();
let conn_clone = conn.clone();
tokio::spawn(async move {
conn_clone.closed().await;
conn.closed().await;
// Any one of them going means the session goes. A client
// left holding audio and input but no video is not a
// degraded session, it is a stuck one, and a clean
// reconnect is both simpler to reason about and quicker
// than whatever partial recovery would be built here.
tracing::debug!(
remote = %remote_id.fmt_short(),
carrier = carrier.label(),
"carrier closed, ending session"
);
mgr_clone.remove_session(&remote_id).await;
});
}
apps/neshub/src/session.rs
+773 -171
View File
File diff suppressed because it is too large Load Diff
apps/nesinit/src/lib.rs
+1
View File
@@ -14,6 +14,7 @@
pub mod filesystems;
pub mod payload;
pub mod platform;
pub mod reap;
pub mod services;
pub mod session;
apps/nesinit/src/main.rs
+2
View File
@@ -45,6 +45,7 @@ fn main() -> anyhow::Result<()> {
// nothing else in this guest is an init system, so until this runs there
// is no `/proc` to score this process in and nowhere to put a socket.
nesinit::filesystems::establish();
nesinit::platform::describe();
// Everything a distribution's init scripts used to do, and nothing else is
// going to: a hostname, the box's address, the directories a session's
@@ -80,6 +81,7 @@ fn main() -> anyhow::Result<()> {
Ok(outcome) => tracing::info!(?outcome, "the session ended"),
Err(error) => tracing::error!(%error, "the session failed"),
}
nesinit::platform::report_steal();
// Before anything below waits on a pid: the reaper runs on this runtime's
// threads, and two things calling `wait` is what the registry exists to
apps/nesinit/src/platform.rs
+148
View File
@@ -0,0 +1,148 @@
// What the kernel under this box actually chose, said once, at debug.
//
// Several of the settings that decide a box's latency are not decided by the
// image. The clocksource is picked at boot and can be demoted by a watchdog,
// the idle driver loads only when the host or the command line asks for it,
// and the preemption model is a boot parameter. None of them is visible from
// outside, and a box has no shell to ask with, so this reads them from sysfs
// and procfs and logs them. Enable with `RUST_LOG=nesinit=debug` on the kernel
// command line.
use std::path::Path;
/// Clocksources a vDSO can read without entering the kernel.
///
/// Anything else makes every `clock_gettime` a syscall, and in a guest an
/// emulated one: `hpet` is an MMIO read the host has to trap. A Windows game
/// polls its performance counter constantly, so that is a cost paid thousands
/// of times a frame, and nothing reports it.
const VDSO_CLOCKSOURCES: &[&str] = &["tsc", "kvm-clock"];
/// Log the kernel's timing and idle choices. Needs `/proc` and `/sys`.
pub fn describe() {
let clocksource = read("/sys/devices/system/clocksource/clocksource0/current_clocksource");
let cmdline = read("/proc/cmdline").unwrap_or_default();
tracing::debug!(
clocksource = clocksource.as_deref().unwrap_or("unknown"),
available = read("/sys/devices/system/clocksource/clocksource0/available_clocksource")
.as_deref()
.unwrap_or("unknown"),
idle_driver = read("/sys/devices/system/cpu/cpuidle/current_driver")
.as_deref()
.unwrap_or("none"),
idle_governor = read("/sys/devices/system/cpu/cpuidle/current_governor_ro")
.as_deref()
.unwrap_or("none"),
// Present only while the haltpoll governor is built in, and only
// meaningful while it is the governor in use.
halt_poll_ns = read("/sys/module/haltpoll/parameters/guest_halt_poll_ns")
.as_deref()
.unwrap_or("n/a"),
// The build's default when absent. The mode actually in effect is only
// readable through debugfs, which this kernel does not have.
preempt = kernel_parameter(&cmdline, "preempt").unwrap_or("default"),
cpus = std::thread::available_parallelism().map_or(0, usize::from),
kernel = read("/proc/sys/kernel/version")
.as_deref()
.unwrap_or("unknown"),
"platform"
);
if let Some(source) = clocksource.as_deref()
&& !VDSO_CLOCKSOURCES.contains(&source)
{
tracing::warn!(
clocksource = source,
"every clock read in this box is a syscall; the kernel did not trust a faster clock"
);
}
}
/// Log how much of this box's CPU time the host took back. Call once, at the
/// end of a session: the counters are cumulative since boot.
pub fn report_steal() {
let Some(stat) = read("/proc/stat") else {
return;
};
let Some((steal, total)) = steal_of(&stat) else {
return;
};
// Parts per thousand, so the log carries an integer and no float
// formatting decides how small a number reads as zero.
let permille = (steal * 1000).checked_div(total).unwrap_or(0);
tracing::debug!(
steal_ticks = steal,
total_ticks = total,
permille,
"cpu time stolen by the host"
);
}
/// Steal and total ticks from the aggregate `cpu` line of `/proc/stat`.
///
/// Columns are user, nice, system, idle, iowait, irq, softirq, steal, and then
/// guest time, which the kernel already counts inside user and nice. Summing
/// past steal would count it twice.
fn steal_of(stat: &str) -> Option<(u64, u64)> {
let line = stat.lines().find(|l| l.starts_with("cpu "))?;
let fields: Vec<u64> = line
.split_whitespace()
.skip(1)
.take(8)
.map(str::parse)
.collect::<Result<_, _>>()
.ok()?;
let steal = *fields.get(7)?;
Some((steal, fields.iter().sum()))
}
/// The value of one bare `name=value` kernel parameter.
fn kernel_parameter<'a>(cmdline: &'a str, name: &str) -> Option<&'a str> {
cmdline
.split_whitespace()
.find_map(|word| word.strip_prefix(name)?.strip_prefix('='))
.filter(|value| !value.is_empty())
}
fn read(path: impl AsRef<Path>) -> Option<String> {
std::fs::read_to_string(path)
.ok()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_parameter_is_found_by_its_whole_name() {
let cmdline = "cpuidle_haltpoll.force=1 console=hvc0 preempt=full ro";
assert_eq!(kernel_parameter(cmdline, "preempt"), Some("full"));
assert_eq!(
kernel_parameter(cmdline, "cpuidle_haltpoll.force"),
Some("1")
);
}
/// `preempt` must not match `preempt_foo=`, which is a different parameter.
#[test]
fn a_longer_name_sharing_the_prefix_is_not_a_match() {
assert_eq!(kernel_parameter("preempt_foo=x", "preempt"), None);
assert_eq!(kernel_parameter("console=hvc0", "preempt"), None);
}
#[test]
fn steal_is_the_eighth_column_and_guest_time_is_not_counted_twice() {
// user nice system idle iowait irq softirq steal guest guest_nice
let stat = "cpu 100 0 50 800 10 5 5 30 999 999\ncpu0 1 2 3 4 5 6 7 8 9 10\n";
assert_eq!(steal_of(stat), Some((30, 1000)));
}
#[test]
fn a_stat_without_a_cpu_line_reports_nothing() {
assert_eq!(steal_of("intr 1 2 3\n"), None);
assert_eq!(steal_of("cpu 1 2 3\n"), None);
}
}
apps/nesinit/src/services.rs
+99 -10
View File
@@ -41,6 +41,7 @@ use tokio::sync::mpsc::{Receiver, Sender};
use crate::reap::{Waiters, Watched};
use crate::workload::Failure;
use nesprotocol::lifecycle::VideoLimits;
/// A service that died, and how.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -60,7 +61,13 @@ pub trait Services {
/// Called once, after the shares are mounted and before anything may be
/// launched. An empty stack is legitimate: a box with no services still
/// boots, and a caller can still launch something that needs none.
fn bring_up(&mut self) -> Result<Vec<String>, Failure>;
///
/// `video` comes from the descriptor and reaches the services that read it.
/// It has to arrive here rather than later because a service configured
/// after it is already running has a window in which it is not configured,
/// and for a bitrate ceiling that window is a session streaming at whatever
/// default it started with.
fn bring_up(&mut self, video: VideoLimits) -> Result<Vec<String>, Failure>;
/// Deaths, as they happen.
///
@@ -160,6 +167,17 @@ pub const SERVICE_UID: u32 = 1000;
/// here: the directory belongs to the service user and is not writable by the
/// workload, which is the property [`crate::ticket::Untrusted`] depends on.
pub const AUDIO_DIR: &str = "/run/pipewire";
/// Where the PulseAudio protocol is served, in [`AUDIO_DIR`] for the reason
/// audio's own socket is.
///
/// pipewire-pulse is told this path by its configuration file in the image,
/// which is not something this program can pass it, so the two are compared by
/// a test rather than trusted to agree.
pub const PULSE_SOCKET: &str = "/run/pipewire/pulse-native";
/// [`PULSE_SOCKET`] as a PulseAudio client is told it.
pub const PULSE_SERVER: &str = "unix:/run/pipewire/pulse-native";
pub const SERVICE_GID: u32 = 1000;
/// Where a service's runtime sockets live.
@@ -273,6 +291,26 @@ pub const STACK: &[Service] = &[
umask: None,
ready: None,
},
Service {
name: "pipewire-pulse",
argv: &["/usr/bin/pipewire-pulse"],
env: &[
("XDG_RUNTIME_DIR", RUNTIME_DIR),
("PIPEWIRE_RUNTIME_DIR", AUDIO_DIR),
("DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"),
],
user: Some((SERVICE_UID, SERVICE_GID)),
// Optional for the reason the sender is: everything that speaks
// PipeWire itself is unaffected, and a session with sound missing is
// degraded rather than unusable.
cost: "anything that only speaks PulseAudio plays silently, and Wine is one",
required: false,
// The workload is its client, and is not this user.
umask: Some(0),
// Nothing in this table connects to it, but the workload does, and the
// workload is started after the table.
ready: Some(PULSE_SOCKET),
},
Service {
name: "neswire",
argv: &["/usr/bin/neswire"],
@@ -308,6 +346,13 @@ pub struct Stack {
running: Vec<(&'static str, Watched)>,
deaths: Receiver<Died>,
reported: Sender<Died>,
/// What the host said this box may spend on video, from the descriptor.
///
/// Held here because `spawn` is where it reaches a service, and `spawn`
/// takes a `&'static Service` whose `env` is a fixed table -- a value that
/// arrives at runtime has no route through it otherwise. The same problem
/// `RUST_LOG` has, solved the same way.
video: VideoLimits,
}
impl Stack {
@@ -328,6 +373,7 @@ impl Stack {
running: Vec::new(),
deaths,
reported,
video: VideoLimits::default(),
}
}
@@ -390,6 +436,15 @@ impl Stack {
if let Ok(filter) = std::env::var("RUST_LOG") {
command.env("RUST_LOG", filter);
}
// The descriptor's video limits, for the services that read them. Same
// shape of problem as `RUST_LOG` above -- `env_clear` drops everything
// and the service table is a fixed list of literals, so a value that
// only exists at runtime has no other route in. `neshub` reads this
// through the clap `env =` attribute it already uses for every other
// setting.
if let Some(kbps) = self.video.bitrate_kbps {
command.env("NESTRI_MAX_BITRATE", kbps.to_string());
}
// The service's own entry last, so a service that states one of these
// for itself wins over the defaults above.
command.envs(service.env.iter().copied());
@@ -451,7 +506,8 @@ impl Stack {
}
impl Services for Stack {
fn bring_up(&mut self) -> Result<Vec<String>, Failure> {
fn bring_up(&mut self, video: VideoLimits) -> Result<Vec<String>, Failure> {
self.video = video;
let mut up = Vec::new();
// Lifted out so the loop does not hold a borrow of `self` across the
// start it is asking for.
@@ -604,6 +660,9 @@ pub mod double {
/// only thing under test.
pub struct Double {
pub brought_up: usize,
/// What the last `bring_up` was told, so a test can assert the limits
/// reached the stack rather than assuming they did.
pub video: VideoLimits,
pub failure: Option<Failure>,
pub names: Vec<String>,
deaths: Receiver<Died>,
@@ -625,6 +684,7 @@ pub mod double {
names: vec!["dbus-system".into(), "neshub".into()],
deaths,
report,
video: VideoLimits::default(),
}
}
@@ -637,8 +697,9 @@ pub mod double {
}
impl Services for Double {
fn bring_up(&mut self) -> Result<Vec<String>, Failure> {
fn bring_up(&mut self, video: VideoLimits) -> Result<Vec<String>, Failure> {
self.brought_up += 1;
self.video = video;
match &self.failure {
Some(failure) => Err(failure.clone()),
None => Ok(self.names.clone()),
@@ -853,14 +914,42 @@ mod tests {
/// owner -- and the workload is not the owner.
#[test]
fn the_audio_socket_is_reachable_by_a_user_who_does_not_own_it() {
let pipewire = STACK
for name in ["pipewire", "pipewire-pulse"] {
let service = STACK
.iter()
.find(|s| s.name == name)
.unwrap_or_else(|| panic!("{name} is in the table"));
assert_eq!(
service.umask,
Some(0),
"with any other umask the game finds {name}'s socket and cannot open it"
);
}
}
/// The PulseAudio socket's path is written three times: here, in the
/// client address the workload is given, and in pipewire-pulse's own
/// configuration in the image. If any of them moves on its own, the game
/// finds no server and plays silently, and nothing fails.
#[test]
fn pulse_is_served_where_the_workload_is_told_to_look() {
assert!(
PULSE_SOCKET.starts_with(AUDIO_DIR),
"the workload can only reach sockets in {AUDIO_DIR}"
);
assert_eq!(PULSE_SERVER, format!("unix:{PULSE_SOCKET}"));
let pulse = STACK
.iter()
.find(|s| s.name == "pipewire")
.expect("audio is in the table");
assert_eq!(
pipewire.umask,
Some(0),
"with any other umask the game finds the socket and cannot open it"
.find(|s| s.name == "pipewire-pulse")
.expect("pulse is in the table");
assert_eq!(pulse.ready, Some(PULSE_SOCKET));
let config =
include_str!("../../../build/etc/pipewire/pipewire-pulse.conf.d/50-nestri.conf");
assert!(
config.contains(&format!("\"{PULSE_SERVER}\"")),
"pipewire-pulse is configured to listen somewhere other than {PULSE_SOCKET}"
);
}
apps/nesinit/src/session.rs
+56 -8
View File
@@ -266,19 +266,19 @@ where
}
booted = true;
// The drives, then the shares, then the services.
// The overlays, then the shares, then the services.
//
// **Drives first, and one `Mounted` between them.** A drive is
// a filesystem this end mounts itself, so a share whose target
// lives under one has to find it already there. The host is
// **Overlays first, and one `Mounted` between them.** An
// overlay is a filesystem this end mounts itself, so a share
// whose target lives under one has to find it already there. The host is
// told once, after both, because `Mounted` answers "is the
// content where the descriptor said" and there is one answer to
// that -- sending it twice made the host read the second as a
// reply to something it had not asked.
if let Err(failure) = workload.mount_drives(&descriptor.drives) {
if let Err(failure) = workload.mount_overlays(&descriptor.overlays) {
// Said before it is returned. `Refused` ends the session
// either way; without the message the host sees a box that
// stopped and has to guess between a drive, a share and a
// stopped and has to guess between an overlay, a share and a
// service -- which is the whole reason these are reported
// separately.
send(
@@ -311,7 +311,7 @@ where
// A box whose own services will not come up cannot be launched
// into, so this is refused rather than reported and carried on
// from — unlike a launch, which is the caller's to correct.
match services.bring_up() {
match services.bring_up(descriptor.video) {
Ok(up) => {
tracing::info!(services = up.len(), "the box is ready to be launched into");
send(&mut writer, &GuestToHost::Initialized { services: up }).await?
@@ -516,7 +516,8 @@ mod tests {
at: "/mnt/user".into(),
ro: false,
}],
drives: Vec::new(),
overlays: Vec::new(),
video: Default::default(),
}
}
@@ -715,6 +716,53 @@ mod tests {
);
}
#[tokio::test]
async fn the_descriptors_video_limits_reach_the_services() {
// The ceiling is useless if it stops at the descriptor. `neshub` is the
// only thing that can enforce it and it is a service, so the number has
// to survive the whole way from the boot document to the spawn.
let (guest, host) = tokio::io::duplex(4096);
let mut caller = Caller::new(host);
let session = spawn(guest, Given::new(Double::exits_when_stopped(Exit::code(0))));
let mut given = descriptor();
given.video.bitrate_kbps = Some(8_000);
caller.expect_ready().await;
caller
.say(&HostToGuest::Boot {
descriptor: Box::new(given),
})
.await;
caller.expect_booted().await;
caller.say(&HostToGuest::Shutdown).await;
let (_, _, services) = session.await.unwrap();
assert_eq!(services.video.bitrate_kbps, Some(8_000));
}
#[tokio::test]
async fn a_box_told_nothing_about_video_says_so_rather_than_inventing_a_limit() {
// "Unsaid" must not arrive as a number. A stack that cannot tell the two
// apart cannot log that it was never told, and a ceiling nobody set is
// exactly how every session came to offer 10 Mbps.
let (guest, host) = tokio::io::duplex(4096);
let mut caller = Caller::new(host);
let session = spawn(guest, Given::new(Double::exits_when_stopped(Exit::code(0))));
caller.expect_ready().await;
caller
.say(&HostToGuest::Boot {
descriptor: Box::new(descriptor()),
})
.await;
caller.expect_booted().await;
caller.say(&HostToGuest::Shutdown).await;
let (_, _, services) = session.await.unwrap();
assert_eq!(services.video.bitrate_kbps, None);
}
#[tokio::test]
async fn a_launch_runs_what_it_names_and_is_reported_by_its_id() {
let (guest, host) = tokio::io::duplex(4096);
apps/nesinit/src/workload.rs
+271 -81
View File
@@ -11,7 +11,7 @@ use std::future::Future;
use std::io;
use std::pin::Pin;
use nesprotocol::lifecycle::{Drive, Exec, Exit, Mount};
use nesprotocol::lifecycle::{Exec, Exit, Mount, Overlay};
use std::os::unix::process::CommandExt;
@@ -41,8 +41,9 @@ pub trait Workload {
/// Make the shares the descriptor names, where it says to put them.
fn mount(&mut self, mounts: &[Mount]) -> Result<(), Failure>;
/// Mount drives
fn mount_drives(&mut self, drives: &[Drive]) -> Result<(), Failure>;
/// Stack each overlay the descriptor names: its build image, its writable
/// layer, and the two together where it says.
fn mount_overlays(&mut self, overlays: &[Overlay]) -> Result<(), Failure>;
/// Start the command the descriptor names.
///
@@ -132,9 +133,9 @@ impl Workload for Process {
Ok(())
}
fn mount_drives(&mut self, drives: &[Drive]) -> Result<(), Failure> {
for drive in drives {
mount_drive(drive)?;
fn mount_overlays(&mut self, overlays: &[Overlay]) -> Result<(), Failure> {
for overlay in overlays {
mount_overlay(overlay)?;
}
Ok(())
}
@@ -302,6 +303,9 @@ const GRAPHICS: &[(&str, &str)] = &[
// reach and both are told where. Without this a game renders and plays
// silently, having looked under its own uid and found nothing.
("PIPEWIRE_RUNTIME_DIR", crate::services::AUDIO_DIR),
// The same, for a client that speaks PulseAudio instead. It does not read
// the variable above, and its default is under its own runtime directory.
("PULSE_SERVER", crate::services::PULSE_SERVER),
];
/// Mount one share where the descriptor says to put it.
@@ -340,35 +344,104 @@ fn mount_share(share: &Mount) -> Result<(), Failure> {
/// filesystem this mounts. A descriptor cannot name another.
const FSTYPE: &std::ffi::CStr = c"virtiofs";
/// Mounts block device instead of virtiofs share
fn mount_drive(drive: &Drive) -> Result<(), Failure> {
// Checked before anything is created: a descriptor this component cannot
// act on should leave no directory behind to confuse whoever reads the
// failure.
let (source, target, flags) = options_drive(drive)?;
/// Stack one overlay: the build image, the box's writable layer, and the two
/// together at `at`.
///
/// Each step's failure names the step, because "the install did not mount"
/// has three different causes and each one is fixed somewhere else: a build
/// image the kernel cannot read, an upper layer that was never formatted, and
/// an overlay the kernel refused.
fn mount_overlay(overlay: &Overlay) -> Result<(), Failure> {
// Everything that can be refused without touching the filesystem is
// refused first, so a descriptor this cannot act on leaves nothing behind.
let plan = OverlayPlan::new(overlay)?;
// The mount point may not exist yet: a share can land anywhere the
// descriptor names, including a directory no image created.
std::fs::create_dir_all(&drive.at).map_err(|error| failed_drive(drive, error))?;
mount_one(
&plan.lower,
&plan.lower_at,
c"erofs",
plan.lower_flags,
None,
)
.map_err(|error| plan.failed("the build image", &plan.lower_at, error))?;
mount_one(&plan.upper, &plan.rw_at, c"ext4", plan.upper_flags, None)
.map_err(|error| plan.failed("the writable layer", &plan.rw_at, error))?;
// SAFETY: mount takes two paths, a filesystem name and a flag word, all
// of which outlive the call, and no options string.
for dir in [&plan.upper_dir, &plan.work_dir] {
std::fs::create_dir_all(as_path(dir))
.map_err(|error| plan.failed("the writable layer", dir, error))?;
}
// **The upper directory takes the build's root ownership.** overlayfs
// shows a merged directory with the attributes of its upper half when it
// has one, and this one always does -- so an upper directory this init
// created, `root:root 0755`, would make the install's top directory
// unwritable to the workload, whatever the build image says. Copied from
// the lower root rather than named here: which uid the workload runs as is
// the host's decision, and the host already made it when it packed the
// image.
let (uid, gid, mode) = ownership(&plan.lower_at)
.map_err(|error| plan.failed("the build image", &plan.lower_at, error))?;
set_ownership(&plan.upper_dir, uid, gid, mode)
.map_err(|error| plan.failed("the writable layer", &plan.upper_dir, error))?;
mount_one(
c"overlay",
&plan.at,
c"overlay",
plan.overlay_flags,
Some(&plan.overlay_data),
)
.map_err(|error| plan.failed("the overlay", &plan.at, error))?;
Ok(())
}
/// Mount one filesystem, creating its mount point first.
fn mount_one(
source: &std::ffi::CStr,
target: &std::ffi::CStr,
fstype: &std::ffi::CStr,
flags: libc::c_ulong,
data: Option<&std::ffi::CStr>,
) -> io::Result<()> {
// The mount point may not exist yet: the descriptor can name anywhere,
// including a directory no image created.
std::fs::create_dir_all(as_path(target))?;
// SAFETY: every pointer is to a nul-terminated string that outlives the
// call, and a null data pointer is what mount(2) takes for "no options".
let mounted = unsafe {
libc::mount(
source.as_ptr(),
target.as_ptr(),
FSTYPE_DRIVE.as_ptr(),
fstype.as_ptr(),
flags,
std::ptr::null(),
data.map_or(std::ptr::null(), |d| d.as_ptr().cast()),
)
};
if mounted != 0 {
return Err(failed_drive(drive, io::Error::last_os_error()));
return Err(io::Error::last_os_error());
}
Ok(())
}
const FSTYPE_DRIVE: &std::ffi::CStr = c"ext4";
/// The same bytes, as a path: lossless, where a round trip through `str` is
/// not.
fn as_path(path: &std::ffi::CStr) -> &std::path::Path {
use std::os::unix::ffi::OsStrExt;
std::path::Path::new(std::ffi::OsStr::from_bytes(path.to_bytes()))
}
fn ownership(path: &std::ffi::CStr) -> io::Result<(u32, u32, u32)> {
use std::os::unix::fs::MetadataExt;
let meta = std::fs::metadata(as_path(path))?;
Ok((meta.uid(), meta.gid(), meta.mode() & 0o7777))
}
fn set_ownership(path: &std::ffi::CStr, uid: u32, gid: u32, mode: u32) -> io::Result<()> {
use std::os::unix::fs::PermissionsExt;
let path = as_path(path);
std::os::unix::fs::chown(path, Some(uid), Some(gid))?;
std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode))
}
/// What the mount call is given, split out because this is the part worth
/// asserting: mounting itself needs privileges a test does not have.
@@ -400,48 +473,114 @@ fn options(share: &Mount) -> Result<(CString, CString, libc::c_ulong), Failure>
Ok((source, target, flags))
}
/// What the drive mount call is given.
/// Everything an overlay's three mounts are given, worked out before any of
/// them is attempted.
///
/// # No filesystem-specific options, and that is a decision
/// Split out because this is the part worth asserting: mounting needs
/// privileges a test does not have.
///
/// `commit=` and `barrier=` were here once and the mount failed outright:
/// *"can't mount with commit=, fs mounted w/o journal"*, `EINVAL`, and a box
/// that refused its own descriptor before the session started. Both options
/// only mean anything to a journal, and a build volume is made without one --
/// what it holds is one game, re-downloadable, mounted by a clone that is
/// destroyed with the box. Anything added here has to be an option that is
/// still true of a journal-less ext4.
/// # Where the layers go
///
/// `noatime` stays: a game reading its own install has no use for access
/// times, and writing them turns every read of a clone into a write. It is not
/// paired with `nodiratime`, which it already implies.
/// Beside the overlay, in a hidden directory named after it:
/// `/nestri/install` stacks `/nestri/.install/lower` under
/// `/nestri/.install/rw/upper`. Beside rather than under, because anything
/// mounted under `at` is covered the moment the overlay is mounted over it.
///
/// # nosuid and nodev, for the same reason every share has them
/// # Flags
///
/// What this mounts is the least trusted thing in the box: files a CDN handed
/// us, checked for the bytes the manifest named and for nothing about what
/// those bytes are. A setuid binary or a device node inside a depot is not
/// something a workload should be able to use, and no descriptor has a way to
/// ask for one.
/// `nosuid` and `nodev` on every layer and on the result, for the reason every
/// share has them: what is stacked here is files a CDN handed us, checked for
/// the bytes the manifest named and for nothing about what those bytes are.
/// **Not `noexec`** anywhere: the game's executable is in the build.
///
/// **Not `noexec`.** The game's own executable is on this volume and the whole
/// point is to run it.
fn options_drive(drive: &Drive) -> Result<(CString, CString, libc::c_ulong), Failure> {
let flags = libc::MS_NOSUID | libc::MS_NODEV | libc::MS_NOATIME;
/// `noatime` on the writable layer and the overlay, so a game reading its own
/// install does not turn every read into a write. The build image is mounted
/// read-only and has no access times to write.
///
/// No filesystem-specific options on the upper layer: `commit=` and
/// `barrier=` were once passed to a journal-less ext4 and the mount failed
/// outright with `EINVAL`.
#[derive(Debug)]
struct OverlayPlan {
lower: CString,
upper: CString,
at: CString,
lower_at: CString,
rw_at: CString,
upper_dir: CString,
work_dir: CString,
lower_flags: libc::c_ulong,
upper_flags: libc::c_ulong,
overlay_flags: libc::c_ulong,
overlay_data: CString,
}
let source = CString::new(drive.dev.as_str()).map_err(|_| {
impl OverlayPlan {
fn new(overlay: &Overlay) -> Result<Self, Failure> {
let at = std::path::Path::new(&overlay.at);
let (Some(parent), Some(name)) = (at.parent(), at.file_name()) else {
return Err(Failure::new(format!(
"the overlay mount point has no parent to put its layers beside: {:?}",
overlay.at
)));
};
let layers = parent.join(format!(".{}", name.to_string_lossy()));
let lower_at = layers.join("lower");
let rw_at = layers.join("rw");
let upper_dir = rw_at.join("upper");
let work_dir = rw_at.join("work");
// overlayfs splits its options on commas and its layer lists on
// colons, and has no escape for either that this should rely on. A
// path carrying one would mount a different directory than the one
// named, so it is refused.
for path in [&lower_at, &upper_dir, &work_dir] {
let text = path.to_string_lossy();
if text.contains([',', ':']) {
return Err(Failure::new(format!(
"the overlay mount point cannot carry a comma or a colon: {:?}",
overlay.at
)));
}
}
let data = format!(
"lowerdir={},upperdir={},workdir={}",
lower_at.display(),
upper_dir.display(),
work_dir.display()
);
let common = libc::MS_NOSUID | libc::MS_NODEV;
Ok(Self {
lower: c_string(&overlay.lower, "the build image device")?,
upper: c_string(&overlay.upper, "the writable layer device")?,
at: c_string(&overlay.at, "the overlay mount point")?,
lower_at: c_string(&lower_at.to_string_lossy(), "the overlay mount point")?,
rw_at: c_string(&rw_at.to_string_lossy(), "the overlay mount point")?,
upper_dir: c_string(&upper_dir.to_string_lossy(), "the overlay mount point")?,
work_dir: c_string(&work_dir.to_string_lossy(), "the overlay mount point")?,
lower_flags: common | libc::MS_RDONLY,
upper_flags: common | libc::MS_NOATIME,
overlay_flags: common | libc::MS_NOATIME,
overlay_data: c_string(&data, "the overlay mount point")?,
})
}
/// Which step failed, on which path, in the operating system's words.
fn failed(&self, step: &str, path: &std::ffi::CStr, error: io::Error) -> Failure {
Failure::new(format!(
"the drive device contains a nul byte: {:?}",
drive.dev
"{}: {step}: {}: {error}",
self.at.to_string_lossy(),
path.to_string_lossy()
))
})?;
let target = CString::new(drive.at.as_str()).map_err(|_| {
Failure::new(format!(
"the drive mount point contains a nul byte: {:?}",
drive.at
))
})?;
Ok((source, target, flags))
}
}
/// A nul byte inside a path is a descriptor that cannot be carried out under
/// any flags. Refused by name rather than silently emptied: an empty path turns
/// up later as a mount failure about something else entirely.
fn c_string(text: &str, what: &str) -> Result<CString, Failure> {
CString::new(text).map_err(|_| Failure::new(format!("{what} contains a nul byte: {text:?}")))
}
/// A failure names the path, which is what makes it actionable: a permission
@@ -451,10 +590,6 @@ fn failed(share: &Mount, error: io::Error) -> Failure {
Failure::new(format!("{}: {error}", share.at))
}
fn failed_drive(drive: &Drive, error: io::Error) -> Failure {
Failure::new(format!("{}: {error}", drive.at))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -578,28 +713,83 @@ mod tests {
assert_eq!(flags & libc::MS_RDONLY, 0);
}
/// The drive carries the same guard every share carries.
///
/// It is the mount that most needs it: a share is a directory this host
/// prepared, and a drive is a filesystem built out of whatever a CDN sent.
fn overlay(at: &str) -> Overlay {
Overlay {
lower: "/dev/vdb".into(),
upper: "/dev/vdc".into(),
at: at.into(),
}
}
/// The layers sit beside the overlay, never under it: anything mounted
/// under `at` is hidden the moment the overlay covers it.
#[test]
fn a_drive_is_mounted_without_devices_or_setuid_but_can_still_execute() {
let drive = Drive {
dev: "/dev/vdb".into(),
at: "/nestri/install".into(),
};
let (source, target, flags) = options_drive(&drive).unwrap();
fn an_overlays_layers_sit_beside_it_and_the_options_name_them() {
let plan = OverlayPlan::new(&overlay("/nestri/install")).unwrap();
assert_eq!(plan.lower.to_str().unwrap(), "/dev/vdb");
assert_eq!(plan.upper.to_str().unwrap(), "/dev/vdc");
assert_eq!(plan.lower_at.to_str().unwrap(), "/nestri/.install/lower");
assert_eq!(plan.rw_at.to_str().unwrap(), "/nestri/.install/rw");
assert_eq!(
source.to_str().unwrap(),
"/dev/vdb",
"the device is the source"
plan.overlay_data.to_str().unwrap(),
"lowerdir=/nestri/.install/lower,\
upperdir=/nestri/.install/rw/upper,\
workdir=/nestri/.install/rw/work"
);
}
/// Every layer carries the guard every share carries, and none of them
/// stops the game's own executable from running.
///
/// These are the mounts that most need it: a share is a directory this
/// host prepared, and a build is a filesystem made out of whatever a CDN
/// sent.
#[test]
fn every_layer_is_mounted_without_devices_or_setuid_but_can_still_execute() {
let plan = OverlayPlan::new(&overlay("/nestri/install")).unwrap();
for flags in [plan.lower_flags, plan.upper_flags, plan.overlay_flags] {
assert_eq!(flags & libc::MS_NOSUID, libc::MS_NOSUID);
assert_eq!(flags & libc::MS_NODEV, libc::MS_NODEV);
assert_eq!(flags & libc::MS_NOEXEC, 0);
}
assert_eq!(plan.lower_flags & libc::MS_RDONLY, libc::MS_RDONLY);
assert_eq!(plan.upper_flags & libc::MS_RDONLY, 0);
assert_eq!(plan.overlay_flags & libc::MS_RDONLY, 0);
assert_eq!(plan.overlay_flags & libc::MS_NOATIME, libc::MS_NOATIME);
}
/// overlayfs splits its options on commas and colons, so a path carrying
/// one would stack a different directory than the one named.
#[test]
fn an_overlay_the_kernel_would_misread_is_refused_before_anything_mounts() {
for at in [
"/nestri/in,stall",
"/nestri/in:stall",
"/",
"/nestri/ins\0tall",
] {
assert!(
OverlayPlan::new(&overlay(at)).is_err(),
"{at:?} was accepted"
);
}
}
#[test]
fn a_failed_overlay_step_names_the_overlay_the_step_and_the_path() {
let plan = OverlayPlan::new(&overlay("/nestri/install")).unwrap();
let failure = plan.failed(
"the build image",
&plan.lower_at,
io::Error::from_raw_os_error(libc::ENODEV),
);
assert!(
failure
.reason
.starts_with("/nestri/install: the build image: /nestri/.install/lower: "),
"{}",
failure.reason
);
assert_eq!(target.to_str().unwrap(), "/nestri/install");
assert_eq!(flags & libc::MS_NOSUID, libc::MS_NOSUID);
assert_eq!(flags & libc::MS_NODEV, libc::MS_NODEV);
assert_eq!(flags & libc::MS_NOATIME, libc::MS_NOATIME);
// The game's executable lives here.
assert_eq!(flags & libc::MS_NOEXEC, 0);
}
#[test]
@@ -652,7 +842,7 @@ pub mod double {
/// only thing under test.
pub struct Double {
pub mounted: Vec<Vec<Mount>>,
pub drives: Vec<Vec<Drive>>,
pub overlays: Vec<Vec<Overlay>>,
pub started: Vec<Exec>,
pub stops: usize,
pub mount_failure: Option<Failure>,
@@ -676,7 +866,7 @@ pub mod double {
fn new(exit: Exit, holds_until_stopped: bool) -> Self {
Self {
mounted: Vec::new(),
drives: Vec::new(),
overlays: Vec::new(),
started: Vec::new(),
stops: 0,
mount_failure: None,
@@ -697,8 +887,8 @@ pub mod double {
}
}
fn mount_drives(&mut self, drives: &[Drive]) -> Result<(), Failure> {
self.drives.push(drives.to_vec());
fn mount_overlays(&mut self, overlays: &[Overlay]) -> Result<(), Failure> {
self.overlays.push(overlays.to_vec());
match &self.mount_failure {
Some(failure) => Err(failure.clone()),
None => Ok(()),
apps/nesinit/tests/services_stop.rs
+3 -1
View File
@@ -50,7 +50,9 @@ fn alive(pid: i32) -> bool {
async fn a_stack_that_goes_away_takes_its_services_with_it() {
let waiters = Waiters::new();
let mut stack = Stack::from_table(waiters, SLEEPERS);
let up = stack.bring_up().expect("two sleeps did not start");
let up = stack
.bring_up(Default::default())
.expect("two sleeps did not start");
assert_eq!(up.len(), 2);
let pids = stack.pids();