mirror of
https://github.com/nestriness/nestri.git
synced 2026-11-03 22:22:39 +02:00
Guest image fixes from #401, plus the dev changes that had not reached prod yet. - **OpenGL games render** (Dome Keeper, confirmed on an AMD host). The guest's Mesa gains its EGL/GLX frontends, and `vm.overcommit_memory=1` stops a large reservation being refused before it is used. - **Chromium / NW.js games get past their first frame** (Sheepy). nescope answers every commit, and paces Xwayland windows too. - **Proton patch step.** `build/patches/proton/` and the `-nN` tag. Nothing uses it until hosts are pointed at the new tag. - Also included: the pixelforge update already on dev. It is in the image tested today. **Versions kept, not downgraded.** Dev still had host agent 0.2.13 and desktop 0.1.0. The last commit restores prod's 0.2.16 and 0.1.2, so this merge changes no install versions. **After merge.** `make push` from this prod commit publishes the guest image. The host agent release that raises LAYOUT to 6 makes hosts pull it. --------- Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
729 lines
41 KiB
Docker
729 lines
41 KiB
Docker
# ═══════════════════════════════════════════════════════════
|
|
# nestri guest rootfs — the open half
|
|
#
|
|
# Builds a bootable Arch image containing Mesa (virtio-gpu native context)
|
|
# and the six open guest components: nesinit, nescope, neshub, neswire,
|
|
# nescapture, nesgamepad. Two leaf targets, selected with `--target`:
|
|
#
|
|
# runtime_prod stripped, root locked (default: `make build`)
|
|
# runtime_debug debug tools, autologin root (`make build-debug`)
|
|
#
|
|
# There is no service manager, no init scripts and no udev. `nesinit` is PID 1
|
|
# and brings the box's services up from a table compiled into it, which is why
|
|
# this image is plain Arch rather than a distribution chosen for its init.
|
|
# ref(d-0064)
|
|
#
|
|
# Proton is NOT in this image any more. It reaches a box as a read-only
|
|
# image the host attaches, so a different one can be tried without
|
|
# rebuilding this. What stays here is everything Proton needs *around* it —
|
|
# its interpreter and its libraries — and the `compat_check` stage below
|
|
# proves a given Proton still has them. Valve's steamclient.so is a
|
|
# different thing and is NOT here either — that one is closed, and
|
|
# nestri/CLAUDE.md is explicit that nothing closed enters this repo.
|
|
# Whatever layers it on top of runtime_prod is a closed build outside this
|
|
# repo — see build/README.md.
|
|
#
|
|
# Build from the repo root, not from build/:
|
|
# docker build -f build/Containerfile --target runtime_prod -t nestri-guest .
|
|
# (`make build` in this directory does exactly that.)
|
|
# ═══════════════════════════════════════════════════════════
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# initial / builder
|
|
#
|
|
# The same distribution the guest is now, which it did not use to be: the
|
|
# guest was Artix, chosen for an init system this image no longer contains.
|
|
# Only build artifacts leave these stages.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM docker.io/archlinux:base-devel AS initial
|
|
RUN pacman -Syu --noconfirm
|
|
|
|
FROM initial AS builder
|
|
RUN pacman -Syu --noconfirm --needed \
|
|
cmake meson ninja git pkgconf mold \
|
|
python python-mako python-yaml python-packaging python-ply \
|
|
bison flex \
|
|
libpciaccess libepoxy libglvnd \
|
|
libx11 libxext libxrandr libxshmfence libxfixes libxxf86vm libxcb \
|
|
xcb-util-keysyms xorgproto \
|
|
wayland wayland-protocols \
|
|
expat zlib zstd libxml2 lm_sensors \
|
|
llvm clang libclc spirv-tools spirv-llvm-translator glslang shader-slang \
|
|
elfutils libva libdrm directx-headers \
|
|
rustup rust-bindgen cbindgen \
|
|
curl openssl \
|
|
pixman libxkbcommon \
|
|
vulkan-headers vulkan-icd-loader \
|
|
pipewire shaderc opus \
|
|
libinput \
|
|
&& pacman -Scc --noconfirm
|
|
|
|
# **Rust comes from rustup, never from the distribution's `rust` package.** That
|
|
# package is built against the distribution's LLVM, and when the two repositories
|
|
# this image draws from disagree about it, `rustc` itself fails to start with a
|
|
# symbol-lookup error — before any of our code is involved, and in every stage
|
|
# that builds Rust. A toolchain of its own cannot be broken by a library it does
|
|
# not link.
|
|
RUN rustup default stable
|
|
|
|
WORKDIR /build
|
|
ENV ARTIFACTS=/artifacts
|
|
ENV RUSTFLAGS="-C link-arg=-fuse-ld=mold"
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# Mesa — the only piece still fetched from outside this tree
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS mesa-build
|
|
|
|
ARG MESA_GIT=https://gitlab.freedesktop.org/mesa/mesa.git
|
|
ARG MESA_COMMIT=a4a47003a194effad8cabe727f74985e9417476f
|
|
ARG JOBS=
|
|
|
|
# OpenGL is zink: GL on top of whichever Vulkan driver the box has, radv over
|
|
# native context or the forwarded NVIDIA one, so one build serves both. Its
|
|
# frontends have to be on for any of it to be reachable. With EGL, GLX, GBM and
|
|
# GLES off, a GL game found no GL at all and ran with sound and a black screen,
|
|
# which is what three alpha testers saw. GLX needs the x11 platform for
|
|
# Xwayland's clients, and Xwayland needs GBM to give them DRI3.
|
|
#
|
|
# Our patches to the amdgpu native-context path, applied on top of the pinned
|
|
# commit. They are not cosmetic: upstream's winsys re-queries device-static
|
|
# facts on a path that costs a synchronous round trip to the host under
|
|
# virtio, and it was most of a frame's time. See each patch's own message.
|
|
COPY build/patches/mesa /build/patches/mesa
|
|
|
|
RUN test -n "$JOBS" || JOBS=$(nproc) && \
|
|
git clone --depth=1 --revision="${MESA_COMMIT}" "${MESA_GIT}" /build/mesa-src && \
|
|
cd /build/mesa-src && \
|
|
git apply --whitespace=nowarn /build/patches/mesa/*.patch && \
|
|
meson setup builddir \
|
|
-Dprefix=/usr \
|
|
-Dbuildtype=release \
|
|
-Dplatforms=wayland,x11 \
|
|
-Dgallium-drivers=zink \
|
|
-Dvulkan-drivers=amd,intel \
|
|
-Damdgpu-virtio=true \
|
|
-Dintel-virtio-experimental=true \
|
|
-Dvideo-codecs=all \
|
|
-Degl=enabled \
|
|
-Dglx=dri \
|
|
-Dglvnd=enabled \
|
|
-Dgles1=disabled \
|
|
-Dgles2=enabled \
|
|
-Dgbm=enabled \
|
|
-Dgallium-va=disabled \
|
|
-Db_ndebug=true && \
|
|
mold -run ninja -C builddir -j${JOBS:-$(nproc)} && \
|
|
DESTDIR=/artifacts/mesa ninja -C builddir install && \
|
|
rm -rf /build/mesa-src && \
|
|
find /artifacts/mesa -type f -printf '/%P\n' > /artifacts/mesa/.manifest
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# nestri workspace — same repo now, so this is COPY, not a private clone
|
|
#
|
|
# One `cargo build --release` over the guest members rather than one stage per
|
|
# binary:
|
|
# that per-repo splitting existed because nescope/neswire/nescapture/the hub
|
|
# were four separate private repos and a stage boundary was the only way to
|
|
# stop bumping one from invalidating the others' build cache. They are one
|
|
# Cargo workspace with one Cargo.lock now, so a BuildKit cache mount on
|
|
# target/ gives the same isolation — cargo's own incremental compiler
|
|
# already knows nescope changing does not touch nesprotocol's .rlib — without
|
|
# four copies of every shared dependency getting compiled once per stage.
|
|
#
|
|
# The members are named rather than `--workspace`, because the workspace holds
|
|
# one crate that is not part of a guest — `nesdoctor` runs on a stranger's own
|
|
# machine — and building it here would compile something this image will never
|
|
# contain. Naming them also means adding a member does not silently add a
|
|
# binary to the image.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS nestri-src
|
|
WORKDIR /build/nestri
|
|
COPY Cargo.toml Cargo.lock ./
|
|
COPY crates/nesprotocol crates/nesprotocol
|
|
COPY crates/nespyro crates/nespyro
|
|
# Not a guest component and not installed below — it runs on a stranger's own
|
|
# machine. It is here because `cargo` loads every workspace member's manifest
|
|
# before it builds anything, so a member missing from the context fails the
|
|
# build outright with `failed to read .../Cargo.toml`. Copying it costs a few
|
|
# files; the member list is the thing that decides, not this build.
|
|
COPY apps/nesdoctor apps/nesdoctor
|
|
COPY apps/nesinit apps/nesinit
|
|
COPY apps/nescope apps/nescope
|
|
COPY apps/neshub apps/neshub
|
|
COPY apps/neswire apps/neswire
|
|
COPY apps/nescapture apps/nescapture
|
|
COPY apps/nesgamepad apps/nesgamepad
|
|
|
|
FROM nestri-src AS nestri-build
|
|
RUN --mount=type=cache,target=/root/.cargo/registry \
|
|
--mount=type=cache,target=/build/nestri/target \
|
|
cargo build --release \
|
|
-p nesinit -p nescope -p neshub -p neswire -p nescapture -p nesgamepad && \
|
|
mkdir -p /artifacts/nestri/usr/bin /artifacts/nestri/usr/lib \
|
|
/artifacts/nestri/usr/share/vulkan/implicit_layer.d && \
|
|
install -Dm755 target/release/nesinit /artifacts/nestri/usr/bin/nesinit && \
|
|
install -Dm755 target/release/nescope /artifacts/nestri/usr/bin/nescope && \
|
|
install -Dm755 target/release/neshub /artifacts/nestri/usr/bin/neshub && \
|
|
install -Dm755 target/release/neswire /artifacts/nestri/usr/bin/neswire && \
|
|
install -Dm755 target/release/nesgamepad /artifacts/nestri/usr/bin/nesgamepad && \
|
|
install -Dm755 target/release/libnescapture_layer.so \
|
|
/artifacts/nestri/usr/lib/libnescapture_layer.so && \
|
|
install -Dm644 apps/nescapture/manifest/VK_LAYER_nescapture.json \
|
|
/artifacts/nestri/usr/share/vulkan/implicit_layer.d/VK_LAYER_nescapture.json && \
|
|
find /artifacts/nestri -type f -printf '/%P\n' > /artifacts/nestri/.manifest
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# virtio-nvgpu's Vulkan layer — for guests on an NVIDIA host
|
|
#
|
|
# On an NVIDIA host the guest runs NVIDIA's own Vulkan driver over a
|
|
# forwarded device, and that driver offers ray tracing, optical flow and a
|
|
# few others it then cannot create a device with: vkCreateDevice fails, and
|
|
# an engine that enables what it is offered does not start. The layer hides
|
|
# them. It acts only on an NVIDIA device in a guest with the virtio-nvgpu
|
|
# driver, so on an AMD or Intel box it is loaded and does nothing.
|
|
#
|
|
# Built from virtio-nvgpu's own tree at NVGPU_REF, the same ref the kernel
|
|
# build takes the guest driver from (build/Makefile), because the layer's job
|
|
# is to say what that driver and its backend can do. The commit is recorded
|
|
# in /usr/share/nestri/virtio-nvgpu.rev.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS nvgpu-layer-build
|
|
ARG NVGPU_GIT=https://github.com/nestrilabs/virtio-nvgpu.git
|
|
ARG NVGPU_REF=dev
|
|
RUN --mount=type=cache,target=/root/.cargo/registry \
|
|
git init -q /build/virtio-nvgpu && cd /build/virtio-nvgpu && \
|
|
git fetch -q --depth=1 "${NVGPU_GIT}" "${NVGPU_REF}" && git checkout -q FETCH_HEAD && \
|
|
cargo build --release -p nvgpu-vklayer && \
|
|
install -Dm755 target/release/libVkLayer_nvgpu.so \
|
|
/artifacts/nvgpu/usr/lib/libVkLayer_nvgpu.so && \
|
|
mkdir -p /artifacts/nvgpu/usr/share/vulkan/implicit_layer.d /artifacts/nvgpu/usr/share/nestri && \
|
|
sed 's|"./libVkLayer_nvgpu.so"|"libVkLayer_nvgpu.so"|' vklayer/VkLayer_nvgpu.json \
|
|
> /artifacts/nvgpu/usr/share/vulkan/implicit_layer.d/VkLayer_nvgpu.json && \
|
|
grep -q '"library_path": "libVkLayer_nvgpu.so"' \
|
|
/artifacts/nvgpu/usr/share/vulkan/implicit_layer.d/VkLayer_nvgpu.json && \
|
|
git rev-parse HEAD > /artifacts/nvgpu/usr/share/nestri/virtio-nvgpu.rev && \
|
|
cd / && rm -rf /build/virtio-nvgpu && \
|
|
find /artifacts/nvgpu -type f -printf '/%P\n' > /artifacts/nvgpu/.manifest
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# virtio-nvgpu's host half — shipped beside the guest's, taken out on the host
|
|
#
|
|
# `vhost-user-nvgpu` serves a box's forwarded GPU and `nvgpu-userspace`
|
|
# stages the host's NVIDIA driver for its guests. They run on the host, never
|
|
# in a guest: the host agent's onboarding moves /boot/host out of this image
|
|
# the way it moves the kernel, and turns forwarding on when it finds the
|
|
# NVIDIA driver loaded. In this image so they are always from the same
|
|
# NVGPU_REF as the guest driver and the layer above.
|
|
#
|
|
# Static (musl), because they run on whatever distribution the host is, and
|
|
# this one's glibc is newer than most of them.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS nvgpu-host-build
|
|
ARG NVGPU_GIT=https://github.com/nestrilabs/virtio-nvgpu.git
|
|
ARG NVGPU_REF=dev
|
|
RUN rustup target add x86_64-unknown-linux-musl
|
|
RUN --mount=type=cache,target=/root/.cargo/registry \
|
|
git init -q /build/virtio-nvgpu && cd /build/virtio-nvgpu && \
|
|
git fetch -q --depth=1 "${NVGPU_GIT}" "${NVGPU_REF}" && git checkout -q FETCH_HEAD && \
|
|
cargo build --release --target x86_64-unknown-linux-musl -p device --features vhost-user \
|
|
--bin vhost-user-nvgpu --bin nvgpu-userspace && \
|
|
for b in vhost-user-nvgpu nvgpu-userspace; do \
|
|
install -Dm755 "target/x86_64-unknown-linux-musl/release/$b" "/artifacts/nvgpu-host/boot/host/$b" && \
|
|
strip "/artifacts/nvgpu-host/boot/host/$b"; \
|
|
done && \
|
|
git rev-parse HEAD > /artifacts/nvgpu-host/boot/host/virtio-nvgpu.rev && \
|
|
cd / && rm -rf /build/virtio-nvgpu
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# The guest kernel — shipped inside the image it boots
|
|
#
|
|
# A host takes it back out as a file beside the root image when it unpacks
|
|
# this one (the host agent's onboarding), because the VMM loads a kernel from the host
|
|
# and never from the guest's disk. Shipping the two as one image is what
|
|
# keeps them a pair: the driver built into this kernel and the Vulkan layer
|
|
# above come from the same NVGPU_REF, and a host can only ever have the
|
|
# kernel that matches its root.
|
|
#
|
|
# The same script `make kernel` runs, on a fresh clone every time: an image
|
|
# build has no warm tree to reuse, and the layer cache already skips this
|
|
# stage while its pins and its inputs are unchanged.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS kernel-build
|
|
ARG KERNEL_GIT=https://github.com/CachyOS/linux.git
|
|
ARG KERNEL_REF=cachyos-7.2.6-1
|
|
ARG KERNEL_MARCH=x86-64-v3
|
|
ARG NVGPU_GIT=https://github.com/nestrilabs/virtio-nvgpu.git
|
|
ARG NVGPU_REF=dev
|
|
RUN pacman -S --noconfirm --needed lld bc cpio perl && pacman -Scc --noconfirm
|
|
COPY build/kernel /kernel-build/kernel
|
|
COPY build/scripts/kernel-build.sh /kernel-build/scripts/kernel-build.sh
|
|
RUN KERNEL_GIT="${KERNEL_GIT}" KERNEL_REF="${KERNEL_REF}" KERNEL_MARCH="${KERNEL_MARCH}" \
|
|
KERNEL_SRC=/build/kernel KERNEL_OUTPUT=/artifacts/kernel/boot/vmlinux \
|
|
NVGPU_GIT="${NVGPU_GIT}" NVGPU_REF="${NVGPU_REF}" NVGPU_WORK=/build/nvgpu \
|
|
bash /kernel-build/scripts/kernel-build.sh && \
|
|
rm -rf /build/kernel /build/nvgpu
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# umu-launcher — built here, not installed from the distribution
|
|
#
|
|
# `umu-run` launches every title that is not Steam's, applying the game fixes
|
|
# Proton carries outside Steam the way launchers such as Heroic do.
|
|
#
|
|
# The distribution's package is not used because of what it declares: it
|
|
# depends on a long list of `lib32-*` libraries and desktop packages that exist
|
|
# for the Steam Linux Runtime container, which this image never runs — Proton
|
|
# runs on the libraries in here (see the compat_check stage). Building from
|
|
# source costs the package and none of that.
|
|
#
|
|
# Built against the distribution's own `urllib3` and `pyzstd` rather than the
|
|
# copies it can vendor: vendoring builds both from source with pinned build
|
|
# tooling, and a rolling distribution moves past those pins (a
|
|
# `setuptools-scm` too new for them failed the build). The image installs the
|
|
# two as ordinary Python packages, with no 32-bit anything.
|
|
#
|
|
# Pinned by tag; it moves deliberately.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS umu-build
|
|
ARG UMU_GIT=https://github.com/Open-Wine-Components/umu-launcher.git
|
|
ARG UMU_TAG=1.4.4
|
|
RUN pacman -Syu --noconfirm --needed \
|
|
make scdoc gcc \
|
|
python-build python-hatchling python-installer python-pip \
|
|
&& pacman -Scc --noconfirm
|
|
RUN git clone --depth 1 --branch "${UMU_TAG}" --recurse-submodules --shallow-submodules \
|
|
"${UMU_GIT}" /umu && \
|
|
cd /umu && \
|
|
./configure.sh --prefix=/usr --use-system-pyzstd --use-system-urllib && \
|
|
make && \
|
|
make DESTDIR=/artifacts/umu install
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# os-base — the Arch rootfs itself
|
|
#
|
|
# `FROM archlinux:base` directly, and `pacman -S` as plain RUN steps — not a
|
|
# privileged host `chroot` into a hand-extracted tarball, which would need
|
|
# /proc, /sys and /dev bind-mounted in first (they don't exist inside a chroot
|
|
# target until something puts them there). A Containerfile RUN step already
|
|
# executes inside a real container with its own /proc, /sys, /dev, so there is
|
|
# no bind-mount step to write at all.
|
|
#
|
|
# # systemd goes; systemd-libs stays
|
|
#
|
|
# Nothing in the package list below depends on `systemd`, and two things in it
|
|
# — dbus-daemon and wireplumber — link `libsystemd.so.0`, which comes from the
|
|
# separate `systemd-libs` package. So the removal is `-Rdd` of `systemd` and
|
|
# `systemd-sysvcompat` only, which is normal rather than a compromise: keeping
|
|
# the library while having no PID 1 from it is exactly how a distribution
|
|
# without systemd ships these same programs.
|
|
#
|
|
# Removing the package also removes its pacman hooks, which is the point. The
|
|
# hooks call `systemd-tmpfiles`, `systemd-sysusers` and `udevadm`; leaving them
|
|
# behind while deleting what they call is how a later transaction fails
|
|
# obscurely, and a previous attempt at this image lost two services to exactly
|
|
# that.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM docker.io/archlinux:base AS os-base
|
|
|
|
# The base image ships an unsigned local keyring, so upgrading
|
|
# `archlinux-keyring` runs a hook that reports `There is no secret key
|
|
# available to sign with` and then `error: command failed to execute
|
|
# correctly`. It is cosmetic and it is also every Arch container's build log.
|
|
# One line fixes it, and it is worth the seconds: a build that always prints an
|
|
# error is a build nobody reads an error out of.
|
|
RUN pacman-key --init
|
|
|
|
# Installed first and removed second, so every dependency resolves normally
|
|
# before anything is taken out from under it.
|
|
#
|
|
# Gone with the init system: `openrc`, `udev`, `dbus-openrc`. `udev` is not
|
|
# replaced by anything — `devtmpfs` creates the nodes and init sets the two
|
|
# modes that matter, because the compositor takes input through Wayland and
|
|
# opens nothing udev provides. ref(d-0064)
|
|
#
|
|
# `logrotate` is also gone, and that one is not about the init system: a box
|
|
# keeps no logs to rotate. What is worth reading leaves over the control
|
|
# channel, and `/var/log` is a small tmpfs that is discarded with the box.
|
|
# `mesa` is not in this list, and the two `--assume-installed` flags are why.
|
|
#
|
|
# The distribution's Mesa used to be installed so that every runtime dependency
|
|
# of *a* Mesa was present and correctly versioned, and ours was then overlaid
|
|
# on top. That worked for the unversioned filenames and not for the versioned
|
|
# one: `libgallium-<version>.so` from the package sat beside ours, 53 MB of it,
|
|
# referenced by nothing. Installing it to overwrite most of it was always the
|
|
# roundabout way round; telling pacman the dependency is already satisfied is
|
|
# the direct one.
|
|
#
|
|
# Exactly two flags are needed and both were checked by dropping each in turn:
|
|
# `mesa` is the name two packages depend on, and `opengl-driver` is a virtual
|
|
# provide `libglvnd` requires that only a real driver package satisfies. The
|
|
# other three names Mesa provides — `mesa-libgl`, `libva-driver`,
|
|
# `libva-mesa-driver` — change nothing here, so they are not listed.
|
|
#
|
|
# What makes this safe is that our Mesa is a superset for this image's
|
|
# purposes: it builds the drivers a box can actually use and the package's
|
|
# other ones (apple, asahi, armada, d3d12) are for hardware no box has. What it
|
|
# does *not* build is a software rasteriser, so there is no llvmpipe fallback —
|
|
# a box with no working GPU path now fails instead of rendering slowly, which
|
|
# is the honest outcome for something that exists to stream frames.
|
|
#
|
|
# Two packages below are explicit *because* Mesa is gone, and both used to
|
|
# arrive as its dependencies: `llvm-libs`, which the radeonsi driver links for
|
|
# shader compilation, and `lm_sensors`, which it links for `libsensors.so.5`.
|
|
# The second was found by the check further down rather than by reading the
|
|
# list — dropping a package takes its dependency tree with it, and the part of
|
|
# that tree something else was quietly using is not visible from here.
|
|
# Created before the install, with the numbers the distribution reserves for them,
|
|
# because the packages hand ownership of a few directories to these names and
|
|
# the hook that would have made the users goes with systemd. `polkit` is here
|
|
# for `rtkit`, which will not grant realtime priority to anything without it;
|
|
# see the services table in `nesinit`.
|
|
RUN groupadd -r -g 102 polkitd && useradd -r -u 102 -g polkitd -d / -s /usr/bin/nologin polkitd && \
|
|
groupadd -r -g 133 rtkit && useradd -r -u 133 -g rtkit -d / -s /usr/bin/nologin rtkit
|
|
|
|
RUN pacman -Syu --noconfirm --needed \
|
|
--assume-installed mesa --assume-installed opengl-driver \
|
|
dbus polkit rtkit \
|
|
iptables iproute2 \
|
|
libglvnd libdrm libepoxy libxxf86vm libinput wayland \
|
|
expat zlib llvm-libs lm_sensors elfutils libva shaderc vulkan-icd-loader \
|
|
pixman libxkbcommon xcb-util-keysyms xorg-xwayland \
|
|
pipewire pipewire-audio wireplumber opus \
|
|
python libunwind sdl2-compat \
|
|
python-xlib python-urllib3 python-pyzstd \
|
|
strace lsof less \
|
|
&& rm -f /usr/share/libalpm/hooks/dbus-reload.hook \
|
|
&& pacman -Rdd --noconfirm systemd systemd-sysvcompat \
|
|
&& pacman -Scc --noconfirm
|
|
|
|
# `strace`, `lsof` and `less` are for the operator's shell (`neslet box exec`):
|
|
# a root shell in a box that cannot trace a process, list what it has open or
|
|
# page through a log is the situation that feature exists to get out of. They
|
|
# are a few hundred kilobytes between them.
|
|
#
|
|
# `libunwind` is Wine's, not ours. `ntdll.so` links it, so without it every
|
|
# process Wine starts dies at `could not load ntdll.so`, which is the first
|
|
# thing it loads and reads like Wine itself being broken. Found 2026-09-12,
|
|
# after the prefix had already been created -- so the session got past every
|
|
# check that Proton was present and usable.
|
|
#
|
|
# `python` is not a build dependency here -- the builder stage has its own for
|
|
# Mesa -- it is a *runtime* one. The compatibility tool's entry point is a
|
|
# Python script, so a box without an interpreter starts a game and the launch
|
|
# ends with `env: 'python3': No such file or directory` and an exit status that
|
|
# reads like an ordinary finish. Found 2026-09-12, on the first session that
|
|
# got as far as launching one.
|
|
|
|
# `sdl2-compat` is Wine's controller support, and the only one there is for a
|
|
# controller that is not a Steam Input one. Proton's device bus reads evdev
|
|
# controllers through SDL2 and nothing else -- its udev backend defers every
|
|
# evdev device to the SDL one on purpose -- and it loads the library with
|
|
# `dlopen`, so its absence fails nothing at build time or at launch. A game
|
|
# simply sees no controller, while `nesgamepad` reports it plugged in. Found
|
|
# 2026-09-27, on the first session with one. Real Steam brings SDL in its
|
|
# runtime, which is why this is never missed anywhere else. On Arch the
|
|
# library is SDL2's API over SDL3, which arrives as its dependency.
|
|
|
|
# `dbus-reload.hook` is deleted above, before the removal rather than after,
|
|
# and it is the whole reason that line is there: the hook runs
|
|
# `/usr/share/libalpm/scripts/systemd-hook`, which systemd owns, so the
|
|
# transaction that removes systemd trips its own leftover on the way out —
|
|
# `call to execv failed`, then `error: command failed to execute correctly`.
|
|
# pacman treats a post-transaction hook failure as non-fatal, so the build
|
|
# survives it and the image is fine; what it leaves is an error message in
|
|
# every future transaction and a reader with no way to tell it from a real
|
|
# one. Removing the hook first means the error never happens.
|
|
|
|
# Nothing left may point at a program that is not here.
|
|
#
|
|
# The specific case above is fixed; this is the general one, and it exists
|
|
# because a hook calling a deleted binary is the exact shape of the failure
|
|
# that took two services off a previous version of this image. A build error
|
|
# is a much better place to find the next one than a log.
|
|
RUN for hook in /usr/share/libalpm/hooks/*.hook; do \
|
|
exec_line="$(awk -F'= *' '/^Exec/ { print $2; exit }' "$hook")"; \
|
|
program="${exec_line%% *}"; \
|
|
case "$program" in /*) ;; *) continue ;; esac; \
|
|
test -e "$program" \
|
|
|| { echo "$(basename "$hook") runs $program, which is not in the image" >&2; exit 1; }; \
|
|
done
|
|
|
|
# The check, because the removal above is the kind of thing a later `pacman
|
|
# -Syu` undoes quietly. A box with systemd's PID 1 back in it boots something
|
|
# other than `nesinit`, and the symptom is a guest that never dials out.
|
|
RUN test ! -e /usr/lib/systemd/systemd \
|
|
|| { echo "systemd's PID 1 is back in the image" >&2; exit 1; }
|
|
|
|
# `groupadd -f`, because some of these already exist in the base image and the
|
|
# rest have to. Nothing creates them at runtime any more: udev's rules did that
|
|
# for device nodes, and with udev gone init sets the two modes that matter
|
|
# directly. ref(d-0064)
|
|
#
|
|
# **Two users, and they must stay two.** `nestri` runs the services that come
|
|
# with this image; `nesplay` is who a workload runs as. Sharing one user between
|
|
# them is what lets workload code impersonate a service — it can replace the
|
|
# socket a service listens on and answer in its place, and the answer that
|
|
# matters is the address a client is told to connect to. Init refuses an address
|
|
# served by the workload's own user, so a single shared user does not merely
|
|
# weaken that check, it makes every session fail it.
|
|
#
|
|
# The uid a workload actually runs as is chosen by whoever asks for the box, not
|
|
# here; this account exists so that the number has a home, a shell and a name in
|
|
# `ps`, and so the separation has somewhere to be written down.
|
|
RUN groupadd -f audio && groupadd -f video && groupadd -f input && groupadd -f render && \
|
|
useradd -m -u 1000 -s /bin/bash nestri && \
|
|
for g in audio video input render; do gpasswd -a nestri "$g" >/dev/null; done && \
|
|
useradd -m -u 1001 -s /bin/bash nesplay && \
|
|
for g in audio video input render; do gpasswd -a nesplay "$g" >/dev/null; done
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# runtime — everything common to debug and prod
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM os-base AS runtime
|
|
|
|
# This is what GHCR actually uses to connect a pushed image back to its
|
|
# repo — not a setting to toggle after the fact, a label the image has to
|
|
# carry. Without it a manually-pushed image shows no "used by" repo on its
|
|
# package page even though this Containerfile is exactly what built it.
|
|
LABEL org.opencontainers.image.source="https://github.com/nestrilabs/nestri"
|
|
|
|
# Our own builds, overlaid on the distro's mesa. The distro package landed
|
|
# first (above) so every runtime dependency of *a* Mesa is present and
|
|
# correctly versioned; this overwrites its .so files with ours.
|
|
#
|
|
# COPY --from runs as root inside this build with no invoking-user uid to
|
|
# stamp onto / or /usr/bin, unlike a host-side `podman cp` + `cp -a` — so
|
|
# there is no ownership-sanity-check to write here. Nothing to catch, on
|
|
# purpose, not an oversight.
|
|
COPY --from=mesa-build /artifacts/mesa/.manifest /tmp/mesa.manifest
|
|
COPY --from=nestri-build /artifacts/nestri/.manifest /tmp/nestri.manifest
|
|
COPY --from=nvgpu-layer-build /artifacts/nvgpu/.manifest /tmp/nvgpu.manifest
|
|
RUN cat /tmp/mesa.manifest /tmp/nestri.manifest /tmp/nvgpu.manifest > /tmp/.strip-manifest && \
|
|
rm -f /tmp/mesa.manifest /tmp/nestri.manifest /tmp/nvgpu.manifest
|
|
COPY --from=mesa-build /artifacts/mesa /
|
|
COPY --from=nestri-build /artifacts/nestri /
|
|
COPY --from=nvgpu-layer-build /artifacts/nvgpu /
|
|
COPY --from=umu-build /artifacts/umu /
|
|
# The kernel this image is booted with, and the host's half of GPU
|
|
# forwarding, for the host to take out. See the kernel-build and
|
|
# nvgpu-host-build stages.
|
|
COPY --from=kernel-build /artifacts/kernel /
|
|
COPY --from=nvgpu-host-build /artifacts/nvgpu-host /
|
|
|
|
# **No Proton here.** It is attached to a box at run time, mounted at
|
|
# /nestri/compat, so this image has no Proton path for a launch to fall back to
|
|
# and a box started without one fails saying where it looked.
|
|
|
|
RUN ldconfig
|
|
|
|
COPY build/etc/ /etc/
|
|
|
|
# `nesinit` is PID 1, and `/usr/bin/init` is the fallback for a kernel started
|
|
# without an explicit `init=`. `systemd-sysvcompat` used to own that path and
|
|
# was removed with the rest of systemd, so nothing else claims it.
|
|
RUN ln -sf nesinit /usr/bin/init
|
|
|
|
# One id per boot, not one per image.
|
|
#
|
|
# `dbus-uuidgen --ensure=/etc/machine-id` used to run here, which baked one id
|
|
# into the image and made every box built from it the same machine. Init writes
|
|
# a fresh one to /run at boot instead, so both of these are symlinks into a
|
|
# tmpfs — which is also the only place they could be, with a read-only root.
|
|
RUN rm -f /etc/machine-id /var/lib/dbus/machine-id && \
|
|
mkdir -p /var/lib/dbus && \
|
|
ln -sf /run/machine-id /etc/machine-id && \
|
|
ln -sf /run/machine-id /var/lib/dbus/machine-id
|
|
|
|
# Session mount points. The guest root is read-only at runtime, so a runtime
|
|
# mkdir gets EROFS and takes a service down before it starts — these have to
|
|
# already exist in the image.
|
|
#
|
|
# `/nestri/logs` is a mount point and nothing mounts it from in here any more:
|
|
# a share arrives because the caller named it in the boot descriptor, which is
|
|
# the same rule every other share follows. The directory stays so that naming
|
|
# it works.
|
|
RUN mkdir -p /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \
|
|
chmod 0755 /nestri /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \
|
|
mkdir -p /dev/shm && chmod 1777 /dev/shm && \
|
|
mkdir -p /run/user/1000 /var/log && \
|
|
# The distribution's own empty `fstab` goes with ours. Nothing in a box
|
|
# reads either: init mounts what a box always needs, and every share comes
|
|
# from the boot descriptor. A file that looks like it configures mounts and
|
|
# is read by nothing is a file somebody edits expecting an effect.
|
|
rm -f /etc/network/interfaces /etc/inittab /etc/fstab
|
|
|
|
# Nothing in this image may be an init system except `nesinit`.
|
|
#
|
|
# A service manager arriving as a dependency of something innocuous is the
|
|
# failure this catches, and it is silent otherwise: the extra init does not run
|
|
# — the kernel is told which one to start — it just sits there with its own
|
|
# ideas about what the box should be doing, waiting for somebody to wire it in.
|
|
RUN for intruder in /usr/lib/systemd/systemd /sbin/openrc-init /usr/bin/openrc-init \
|
|
/sbin/runit-init /usr/bin/runit-init /sbin/dinit /usr/bin/dinit; do \
|
|
test ! -e "$intruder" || { echo "a second init is in the image: $intruder" >&2; exit 1; }; \
|
|
done
|
|
|
|
# What `nesinit` will look for at runtime, checked while there is somebody to
|
|
# read the failure.
|
|
#
|
|
# It is a table compiled into a binary, so a missing program is not a build
|
|
# error — it is a service that does not come up in a box somebody is waiting
|
|
# on, reported over the control channel and read hours later. Checking here
|
|
# turns that into a failed build.
|
|
# Everything this image promises must resolve the libraries it links.
|
|
#
|
|
# This is the check the Mesa change needs: dropping a package that provided
|
|
# libraries is how a binary ends up resolving nothing, and the symptom is not a
|
|
# build failure — it is a service that will not start in a box somebody is
|
|
# waiting on, or a render path that is missing rather than slow. It caught
|
|
# exactly that on the first run, and the missing library was two levels down a
|
|
# dependency tree nobody had reason to read.
|
|
#
|
|
# **Named rather than swept, and that is deliberate.** A sweep over everything
|
|
# in /usr/lib fails on a stock image: a distribution ships optional plugins
|
|
# whose optional dependencies are not installed — pinentry's Qt build, mpg123's
|
|
# JACK output, libdecor's GTK backend — and every one of those was already
|
|
# unresolved before this stage existed. A check that reports a dozen things
|
|
# nobody intends to load is a check the next person deletes. This list is what
|
|
# the image is *for*: the components, the services init starts, and the chain
|
|
# between a workload and the GPU. Wine's own core is checked by the
|
|
# `compat_check` stage, against the Proton it is actually going to run.
|
|
#
|
|
# **Wine was added after it was missed**, and then narrowed twice, which is
|
|
# worth recording so nobody widens it again.
|
|
#
|
|
# It was missed because the list covered everything this image ships *of ours*
|
|
# and nothing of the compatibility tool's, so an unresolved `libunwind.so.8`
|
|
# behind `ntdll.so` survived a build whose whole purpose is catching that, and
|
|
# surfaced as a session that created a prefix and could not start one process
|
|
# in it.
|
|
#
|
|
# The obvious fix -- sweep every `*-unix/*.so` -- is wrong in both directions.
|
|
# It is noisy: those objects are Wine's optional backends, and their
|
|
# dependencies are a camera library, a media stack, a VR loader, a smartcard
|
|
# daemon and OpenCL, none of which belong in a box. And it cannot see what it
|
|
# is checking: Wine's unix objects **link each other by soname** and are
|
|
# resolved by Wine's own loader rather than by `ld.so`, so `ldd` reports
|
|
# `ntdll.so` and `win32u.so` themselves as missing while they sit in the same
|
|
# directory. Forty files, every one a false positive, hiding the one real
|
|
# entry.
|
|
#
|
|
# So: the programs in `bin/`, which are ordinary ELF and resolve normally,
|
|
# `ntdll.so`, which is the first thing Wine loads and the one that linked the
|
|
# missing library, and `winepulse.so`, which is audio. That is exactly the
|
|
# failure that got through, with none of the noise. The Windows-side DLLs beside
|
|
# them are not ELF and `ldd` skips them anyway.
|
|
#
|
|
# Each file is resolved with its own directory on the library path. That is
|
|
# where Wine's loader finds the siblings a unix object links by soname, so
|
|
# `winepulse.so` naming `ntdll.so` resolves the way it does at run time, and
|
|
# what is left is a library the image really lacks. For every other file in
|
|
# the list it changes nothing.
|
|
# The output is one file per line with its own missing libraries under it, and
|
|
# then every missing library once at the end. That last list is what somebody
|
|
# acts on -- it is the set of packages to add -- and forty files each naming the
|
|
# same two libraries is not that list. An earlier version printed one
|
|
# comma-joined line and cut the wrong field out of `ldd`, so it named no
|
|
# libraries at all: `ldd` indents with a tab, which `tr -s ' '` does not
|
|
# collapse, so the second space-separated field is `=>`.
|
|
RUN failed=0; \
|
|
: > /tmp/missing-libs; \
|
|
for f in /usr/bin/nesinit /usr/bin/nescope /usr/bin/neshub /usr/bin/neswire \
|
|
/usr/bin/nesgamepad /usr/lib/libnescapture_layer.so /usr/lib/libSDL2-2.0.so.0 \
|
|
/usr/lib/libVkLayer_nvgpu.so \
|
|
/usr/bin/dbus-daemon /usr/bin/pipewire \
|
|
/usr/lib/polkit-1/polkitd /usr/lib/rtkit-daemon \
|
|
/usr/bin/wireplumber /usr/bin/ip \
|
|
/usr/lib/libgallium-*.so /usr/lib/libEGL_mesa.so.0 \
|
|
/usr/lib/libvulkan_*.so /usr/lib/dri/*.so /usr/lib/gbm/*.so; do \
|
|
[ -e "$f" ] || continue; \
|
|
libs="$(LD_LIBRARY_PATH="${f%/*}" ldd "$f" 2>/dev/null | awk '/not found/ { print $1 }')"; \
|
|
[ -n "$libs" ] || continue; \
|
|
failed=1; \
|
|
printf ' %s\n' "$f" >&2; \
|
|
printf ' %s\n' $libs >&2; \
|
|
printf '%s\n' $libs >> /tmp/missing-libs; \
|
|
done; \
|
|
if [ "$failed" != 0 ]; then \
|
|
echo "" >&2; \
|
|
echo " every library above, once each -- this is the list to install:" >&2; \
|
|
sort -u /tmp/missing-libs | sed 's/^/ /' >&2; \
|
|
exit 1; \
|
|
fi
|
|
|
|
RUN for required in /usr/bin/nesinit /usr/bin/nescope /usr/bin/neshub /usr/bin/neswire \
|
|
/usr/bin/nesgamepad /usr/lib/libSDL2-2.0.so.0 \
|
|
/usr/bin/dbus-daemon /usr/bin/pipewire \
|
|
/usr/lib/polkit-1/polkitd /usr/lib/rtkit-daemon \
|
|
/usr/bin/wireplumber /usr/bin/ip /usr/bin/python3 /usr/bin/umu-run; do \
|
|
test -x "$required" || { echo "the image is missing $required" >&2; exit 1; }; \
|
|
done
|
|
|
|
# `umu-run` has to start, not only exist. It is a Python program built in another
|
|
# stage: a module it imports that this image lacks, or an interpreter that moved
|
|
# a minor version between that stage and this one, is a game that never starts —
|
|
# and the launch reports a failure to run a script, not a missing module.
|
|
RUN umu-run --help >/dev/null 2>&1 \
|
|
|| { echo "umu-run does not start; run it by hand to see the module or interpreter it lacks" >&2; \
|
|
umu-run --help >&2 || true; exit 1; }
|
|
|
|
# No 32-bit libraries came with it. The distribution's umu package would have
|
|
# brought a long list of them for the container runtime this image never runs,
|
|
# which is why it is built instead of installed; this is the check that stays
|
|
# true if somebody swaps it back.
|
|
RUN ! pacman -Qq | grep -q '^lib32-' \
|
|
|| { echo "a lib32 package is installed:" >&2; pacman -Qq | grep '^lib32-' >&2; exit 1; }
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# runtime_prod — the default: `make build`
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM runtime AS runtime_prod
|
|
|
|
# No console is offered by either flavour: `nesinit` spawns no getty, because
|
|
# the way into a guest that will not boot is `init=/bin/bash` on the kernel
|
|
# command line, which needs nothing from the image but a shell. So the locked
|
|
# root account is belt and braces rather than the only thing standing between
|
|
# a box and a login prompt.
|
|
RUN passwd -l root
|
|
|
|
RUN while IFS= read -r f; do \
|
|
[ -f "$f" ] && strip --strip-unneeded "$f" 2>/dev/null || true; \
|
|
done < /tmp/.strip-manifest; \
|
|
rm -rf /tmp/.strip-manifest /var/cache/pacman/pkg/* /tmp/* /root/.cache \
|
|
/usr/share/man /usr/share/doc /usr/share/locale \
|
|
/usr/lib/cmake /usr/lib/pkgconfig /usr/share/pkgconfig /usr/include \
|
|
/usr/share/gir-1.0 /usr/lib/udev; \
|
|
find /usr/lib -name '*.a' -delete
|
|
|
|
RUN echo "NESTRI_STAGE=runtime_prod" >> /etc/os-release
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# runtime_debug — `make build-debug`
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM runtime AS runtime_debug
|
|
|
|
# `-Sy` and not `-Syu`: a full upgrade here can pull a package back in as
|
|
# somebody's dependency, and the one that matters is systemd. The check below
|
|
# catches it either way, but a debug image that fails to build is worse than
|
|
# one that is a few days behind on versions it only uses for `vulkaninfo`.
|
|
RUN pacman -Sy --noconfirm --needed vulkan-tools mesa-utils libva-utils && \
|
|
pacman -Scc --noconfirm
|
|
|
|
# The same guard as the runtime stage, because the transaction above is exactly
|
|
# the kind that quietly reinstates an init system.
|
|
RUN test ! -e /usr/lib/systemd/systemd \
|
|
|| { echo "systemd's PID 1 came back with the debug tools" >&2; exit 1; }
|
|
|
|
# Root has a password here and nothing offers a login prompt to type it into.
|
|
# It is for `su` from an `init=/bin/bash` shell, which is the whole debug route.
|
|
RUN echo 'root:nestri' | chpasswd
|
|
|
|
RUN echo "NESTRI_STAGE=runtime_debug" >> /etc/os-release
|