github-mirrors/netris-nestri
1
0
Fork 0
mirror of https://github.com/nestriness/nestri.git synced 2025-12-12 16:55:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: Controller support, performance enchancements, multi-stage images, fixes (#304)

Browse Source 
## Description
Oops.. another massive PR 🥲 

This PR contains multiple improvements and changes.

Firstly, thanks gst-wayland-display's PR
[here](https://github.com/games-on-whales/gst-wayland-display/pull/20).
NVIDIA path is now way more efficient than before.

Secondly, adding controller support was a massive hurdle, requiring me
to start another project
[vimputti](https://github.com/DatCaptainHorse/vimputti) - which allows
simple virtual controller inputs in isolated containers. Well, it's not
simple, it includes LD_PRELOAD shims and other craziness, but the
library API is simple to use..

Thirdly, split runner image into 3 separate stages, base + build +
runtime, should help keep things in check in future, also added GitHub
Actions CI builds for v2 to v4 builds (hopefully they pass..).

Fourth, replaced the runner's runtime Steam patching with better and
simpler bubblewrap patch, massive thanks to `games-on-whales` to
figuring it out better!

Fifth, relay for once needed some changes, the new changes are still
mostly WIP, but I'll deal with them next time I have energy.. I'm spent
now. Needed to include these changes as relay needed a minor change to
allow rumble events to flow back to client peer.

Sixth.. tons of package updates, minor code improvements and the usual. 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* End-to-end gamepad/controller support (attach/detach, buttons, sticks,
triggers, rumble) with client/server integration and virtual controller
plumbing.
  * Optional Prometheus metrics endpoint and WebTransport support.
  * Background vimputti manager process added for controller handling.

* **Improvements**
  * Multi-variant container image builds and streamlined runtime images.
  * Zero-copy video pipeline and encoder improvements for lower latency.
  * Updated Steam compat mapping and dependency/toolchain refreshes.

* **Bug Fixes**
* More robust GPU detection, input/fullscreen lifecycle,
startup/entrypoint, and container runtime fixes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
This commit is contained in:
Kristian Ollikainen
2025-10-20 11:20:05 +03:00
committed by GitHub
parent a3ee9aadd9
commit c62a22b552
62 changed files with 4203 additions and 2278 deletions
Download Patch File Download Diff File Expand all files Collapse all files

143
packages/server/src/enc_helper.rs
View File

@@ -276,8 +276,8 @@ pub fn encoder_low_latency_params(
_rate_control: &RateControl,
framerate: u32,
) -> VideoEncoderInfo {
// 2 second GOP size, maybe lower to 1 second for fast recovery, if needed?
let mut encoder_optz = encoder_gop_params(encoder, framerate * 2);
// 1 second keyframe interval for fast recovery, is this too taxing?
let mut encoder_optz = encoder_gop_params(encoder, framerate);
match encoder_optz.encoder_api {
EncoderAPI::QSV => {
@@ -291,6 +291,7 @@ pub fn encoder_low_latency_params(
encoder_optz.set_parameter("multi-pass", "disabled");
encoder_optz.set_parameter("preset", "p1");
encoder_optz.set_parameter("tune", "ultra-low-latency");
encoder_optz.set_parameter("zerolatency", "true");
}
EncoderAPI::AMF => {
encoder_optz.set_parameter("preset", "speed");
@@ -400,11 +401,21 @@ pub fn get_compatible_encoders(gpus: &Vec<GPUInfo>) -> Vec<VideoEncoderInfo> {
}
None
} else if element.has_property("cuda-device-id") {
let device_id =
match element.property_value("cuda-device-id").get::<i32>() {
Ok(v) if v >= 0 => Some(v as usize),
_ => None,
};
let device_id = match element
.property_value("cuda-device-id")
.get::<i32>()
{
Ok(v) if v >= 0 => Some(v as usize),
_ => {
// If only one NVIDIA GPU, default to 0
// fixes "Type: 'Hardware', Device: 'CPU'" issue
if get_gpus_by_vendor(&gpus, GPUVendor::NVIDIA).len() == 1 {
Some(0)
} else {
None
}
}
};
// We'll just treat cuda-device-id as an index
device_id.and_then(|id| {
@@ -574,7 +585,7 @@ pub fn get_best_working_encoder(
encoders: &Vec<VideoEncoderInfo>,
codec: &Codec,
encoder_type: &EncoderType,
dma_buf: bool,
zero_copy: bool,
) -> Result<VideoEncoderInfo, Box<dyn Error>> {
let mut candidates = get_encoders_by_videocodec(
encoders,
@@ -590,7 +601,7 @@ pub fn get_best_working_encoder(
while !candidates.is_empty() {
let best = get_best_compatible_encoder(&candidates, codec, encoder_type)?;
tracing::info!("Testing encoder: {}", best.name,);
if test_encoder(&best, dma_buf).is_ok() {
if test_encoder(&best, zero_copy).is_ok() {
return Ok(best);
} else {
// Remove this encoder and try next best
@@ -602,7 +613,7 @@ pub fn get_best_working_encoder(
}
/// Test if a pipeline with the given encoder can be created and set to Playing
pub fn test_encoder(encoder: &VideoEncoderInfo, dma_buf: bool) -> Result<(), Box<dyn Error>> {
pub fn test_encoder(encoder: &VideoEncoderInfo, zero_copy: bool) -> Result<(), Box<dyn Error>> {
let src = gstreamer::ElementFactory::make("waylanddisplaysrc").build()?;
if let Some(gpu_info) = &encoder.gpu_info {
src.set_property_from_str("render-node", gpu_info.render_path());
@@ -610,12 +621,16 @@ pub fn test_encoder(encoder: &VideoEncoderInfo, dma_buf: bool) -> Result<(), Box
let caps_filter = gstreamer::ElementFactory::make("capsfilter").build()?;
let caps = gstreamer::Caps::from_str(&format!(
"{},width=1280,height=720,framerate=30/1{}",
if dma_buf {
"video/x-raw(memory:DMABuf)"
if zero_copy {
if encoder.encoder_api == EncoderAPI::NVENC {
"video/x-raw(memory:CUDAMemory)"
} else {
"video/x-raw(memory:DMABuf)"
}
} else {
"video/x-raw"
},
if dma_buf { "" } else { ",format=RGBx" }
if zero_copy { "" } else { ",format=RGBx" }
))?;
caps_filter.set_property("caps", &caps);
@@ -627,66 +642,47 @@ pub fn test_encoder(encoder: &VideoEncoderInfo, dma_buf: bool) -> Result<(), Box
// Create pipeline and link elements
let pipeline = gstreamer::Pipeline::new();
if dma_buf && encoder.encoder_api == EncoderAPI::NVENC {
// GL upload element
let glupload = gstreamer::ElementFactory::make("glupload").build()?;
// GL color convert element
let glconvert = gstreamer::ElementFactory::make("glcolorconvert").build()?;
// GL color convert caps
let gl_caps_filter = gstreamer::ElementFactory::make("capsfilter").build()?;
let gl_caps = gstreamer::Caps::from_str("video/x-raw(memory:GLMemory),format=NV12")?;
gl_caps_filter.set_property("caps", &gl_caps);
// CUDA upload element
let cudaupload = gstreamer::ElementFactory::make("cudaupload").build()?;
if zero_copy {
if encoder.encoder_api == EncoderAPI::NVENC {
// NVENC zero-copy path
pipeline.add_many(&[&src, &caps_filter, &enc, &sink])?;
gstreamer::Element::link_many(&[&src, &caps_filter, &enc, &sink])?;
} else {
// VA-API/QSV zero-copy path
let vapostproc = gstreamer::ElementFactory::make("vapostproc").build()?;
let va_caps_filter = gstreamer::ElementFactory::make("capsfilter").build()?;
let va_caps = gstreamer::Caps::from_str("video/x-raw(memory:VAMemory),format=NV12")?;
va_caps_filter.set_property("caps", &va_caps);
pipeline.add_many(&[
&src,
&caps_filter,
&glupload,
&glconvert,
&gl_caps_filter,
&cudaupload,
&enc,
&sink,
])?;
gstreamer::Element::link_many(&[
&src,
&caps_filter,
&glupload,
&glconvert,
&gl_caps_filter,
&cudaupload,
&enc,
&sink,
])?;
pipeline.add_many(&[
&src,
&caps_filter,
&vapostproc,
&va_caps_filter,
&enc,
&sink,
])?;
gstreamer::Element::link_many(&[
&src,
&caps_filter,
&vapostproc,
&va_caps_filter,
&enc,
&sink,
])?;
}
} else {
let vapostproc = gstreamer::ElementFactory::make("vapostproc").build()?;
// VA caps filter
let va_caps_filter = gstreamer::ElementFactory::make("capsfilter").build()?;
let va_caps = gstreamer::Caps::from_str("video/x-raw(memory:VAMemory),format=NV12")?;
va_caps_filter.set_property("caps", &va_caps);
pipeline.add_many(&[
&src,
&caps_filter,
&vapostproc,
&va_caps_filter,
&enc,
&sink,
])?;
gstreamer::Element::link_many(&[
&src,
&caps_filter,
&vapostproc,
&va_caps_filter,
&enc,
&sink,
])?;
// Non-zero-copy path for all encoders - needs videoconvert
let videoconvert = gstreamer::ElementFactory::make("videoconvert").build()?;
pipeline.add_many(&[&src, &caps_filter, &videoconvert, &enc, &sink])?;
gstreamer::Element::link_many(&[&src, &caps_filter, &videoconvert, &enc, &sink])?;
}
let bus = pipeline.bus().ok_or("Pipeline has no bus")?;
let _ = pipeline.set_state(gstreamer::State::Playing);
for msg in bus.iter_timed(gstreamer::ClockTime::from_seconds(2)) {
pipeline.set_state(gstreamer::State::Playing)?;
// Wait for either error or async-done (state change complete)
for msg in bus.iter_timed(gstreamer::ClockTime::from_seconds(10)) {
match msg.view() {
gstreamer::MessageView::Error(err) => {
let err_msg = format!("Pipeline error: {}", err.error());
@@ -694,14 +690,17 @@ pub fn test_encoder(encoder: &VideoEncoderInfo, dma_buf: bool) -> Result<(), Box
let _ = pipeline.set_state(gstreamer::State::Null);
return Err(err_msg.into());
}
gstreamer::MessageView::Eos(_) => {
tracing::info!("Pipeline EOS received");
gstreamer::MessageView::AsyncDone(_) => {
// Pipeline successfully reached PLAYING state
tracing::debug!("Pipeline reached PLAYING state successfully");
let _ = pipeline.set_state(gstreamer::State::Null);
return Err("Pipeline EOS received, encoder test failed".into());
return Ok(());
}
_ => {}
}
}
// If we got here, timeout occurred without reaching PLAYING or error
let _ = pipeline.set_state(gstreamer::State::Null);
Ok(())
Err("Encoder test timed out".into())
}