mirror of
https://github.com/nestriness/nestri.git
synced 2026-11-03 22:22:39 +02:00
Wanjohi
bf65df7e1a
- **Guest image (#405):** carries `libvpx`, `speex`, `libtheora`, `libvdpau` and `jsoncpp`, which Proton's GStreamer plugins link. Without them, `libgstlibav` and `libgstspeex` failed to load in every box on every Proton shipped so far. Verified: every library both published Protons need is now provided. - **Billing (#406):** every Polar webhook logs its type, the team it resolved to, the plan and status, whether it changed anything, and the subscription and customer ids. Logging only. Cut from prod rather than opened from dev: after squash merges the two histories conflict, though the files differ only in these four. This branch's tree is identical to dev's. After merge, the guest image is published from this prod commit. Host agent 0.2.18 then makes hosts pull it.
740 lines
42 KiB
Docker
740 lines
42 KiB
Docker
# ═══════════════════════════════════════════════════════════
|
|
# nestri guest rootfs — the open half
|
|
#
|
|
# Builds a bootable Arch image containing Mesa (virtio-gpu native context)
|
|
# and the six open guest components: nesinit, nescope, neshub, neswire,
|
|
# nescapture, nesgamepad. Two leaf targets, selected with `--target`:
|
|
#
|
|
# runtime_prod stripped, root locked (default: `make build`)
|
|
# runtime_debug debug tools, autologin root (`make build-debug`)
|
|
#
|
|
# There is no service manager, no init scripts and no udev. `nesinit` is PID 1
|
|
# and brings the box's services up from a table compiled into it, which is why
|
|
# this image is plain Arch rather than a distribution chosen for its init.
|
|
# ref(d-0064)
|
|
#
|
|
# Proton is NOT in this image any more. It reaches a box as a read-only
|
|
# image the host attaches, so a different one can be tried without
|
|
# rebuilding this. What stays here is everything Proton needs *around* it —
|
|
# its interpreter and its libraries — and the `compat_check` stage below
|
|
# proves a given Proton still has them. Valve's steamclient.so is a
|
|
# different thing and is NOT here either — that one is closed, and
|
|
# nestri/CLAUDE.md is explicit that nothing closed enters this repo.
|
|
# Whatever layers it on top of runtime_prod is a closed build outside this
|
|
# repo — see build/README.md.
|
|
#
|
|
# Build from the repo root, not from build/:
|
|
# docker build -f build/Containerfile --target runtime_prod -t nestri-guest .
|
|
# (`make build` in this directory does exactly that.)
|
|
# ═══════════════════════════════════════════════════════════
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# initial / builder
|
|
#
|
|
# The same distribution the guest is now, which it did not use to be: the
|
|
# guest was Artix, chosen for an init system this image no longer contains.
|
|
# Only build artifacts leave these stages.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM docker.io/archlinux:base-devel AS initial
|
|
RUN pacman -Syu --noconfirm
|
|
|
|
FROM initial AS builder
|
|
RUN pacman -Syu --noconfirm --needed \
|
|
cmake meson ninja git pkgconf mold \
|
|
python python-mako python-yaml python-packaging python-ply \
|
|
bison flex \
|
|
libpciaccess libepoxy libglvnd \
|
|
libx11 libxext libxrandr libxshmfence libxfixes libxxf86vm libxcb \
|
|
xcb-util-keysyms xorgproto \
|
|
wayland wayland-protocols \
|
|
expat zlib zstd libxml2 lm_sensors \
|
|
llvm clang libclc spirv-tools spirv-llvm-translator glslang shader-slang \
|
|
elfutils libva libdrm directx-headers \
|
|
rustup rust-bindgen cbindgen \
|
|
curl openssl \
|
|
pixman libxkbcommon \
|
|
vulkan-headers vulkan-icd-loader \
|
|
pipewire shaderc opus \
|
|
libinput \
|
|
&& pacman -Scc --noconfirm
|
|
|
|
# **Rust comes from rustup, never from the distribution's `rust` package.** That
|
|
# package is built against the distribution's LLVM, and when the two repositories
|
|
# this image draws from disagree about it, `rustc` itself fails to start with a
|
|
# symbol-lookup error — before any of our code is involved, and in every stage
|
|
# that builds Rust. A toolchain of its own cannot be broken by a library it does
|
|
# not link.
|
|
RUN rustup default stable
|
|
|
|
WORKDIR /build
|
|
ENV ARTIFACTS=/artifacts
|
|
ENV RUSTFLAGS="-C link-arg=-fuse-ld=mold"
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# Mesa — the only piece still fetched from outside this tree
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS mesa-build
|
|
|
|
ARG MESA_GIT=https://gitlab.freedesktop.org/mesa/mesa.git
|
|
ARG MESA_COMMIT=a4a47003a194effad8cabe727f74985e9417476f
|
|
ARG JOBS=
|
|
|
|
# OpenGL is zink: GL on top of whichever Vulkan driver the box has, radv over
|
|
# native context or the forwarded NVIDIA one, so one build serves both. Its
|
|
# frontends have to be on for any of it to be reachable. With EGL, GLX, GBM and
|
|
# GLES off, a GL game found no GL at all and ran with sound and a black screen,
|
|
# which is what three alpha testers saw. GLX needs the x11 platform for
|
|
# Xwayland's clients, and Xwayland needs GBM to give them DRI3.
|
|
#
|
|
# Our patches to the amdgpu native-context path, applied on top of the pinned
|
|
# commit. They are not cosmetic: upstream's winsys re-queries device-static
|
|
# facts on a path that costs a synchronous round trip to the host under
|
|
# virtio, and it was most of a frame's time. See each patch's own message.
|
|
COPY build/patches/mesa /build/patches/mesa
|
|
|
|
RUN test -n "$JOBS" || JOBS=$(nproc) && \
|
|
git clone --depth=1 --revision="${MESA_COMMIT}" "${MESA_GIT}" /build/mesa-src && \
|
|
cd /build/mesa-src && \
|
|
git apply --whitespace=nowarn /build/patches/mesa/*.patch && \
|
|
meson setup builddir \
|
|
-Dprefix=/usr \
|
|
-Dbuildtype=release \
|
|
-Dplatforms=wayland,x11 \
|
|
-Dgallium-drivers=zink \
|
|
-Dvulkan-drivers=amd,intel \
|
|
-Damdgpu-virtio=true \
|
|
-Dintel-virtio-experimental=true \
|
|
-Dvideo-codecs=all \
|
|
-Degl=enabled \
|
|
-Dglx=dri \
|
|
-Dglvnd=enabled \
|
|
-Dgles1=disabled \
|
|
-Dgles2=enabled \
|
|
-Dgbm=enabled \
|
|
-Dgallium-va=disabled \
|
|
-Db_ndebug=true && \
|
|
mold -run ninja -C builddir -j${JOBS:-$(nproc)} && \
|
|
DESTDIR=/artifacts/mesa ninja -C builddir install && \
|
|
rm -rf /build/mesa-src && \
|
|
find /artifacts/mesa -type f -printf '/%P\n' > /artifacts/mesa/.manifest
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# nestri workspace — same repo now, so this is COPY, not a private clone
|
|
#
|
|
# One `cargo build --release` over the guest members rather than one stage per
|
|
# binary:
|
|
# that per-repo splitting existed because nescope/neswire/nescapture/the hub
|
|
# were four separate private repos and a stage boundary was the only way to
|
|
# stop bumping one from invalidating the others' build cache. They are one
|
|
# Cargo workspace with one Cargo.lock now, so a BuildKit cache mount on
|
|
# target/ gives the same isolation — cargo's own incremental compiler
|
|
# already knows nescope changing does not touch nesprotocol's .rlib — without
|
|
# four copies of every shared dependency getting compiled once per stage.
|
|
#
|
|
# The members are named rather than `--workspace`, because the workspace holds
|
|
# one crate that is not part of a guest — `nesdoctor` runs on a stranger's own
|
|
# machine — and building it here would compile something this image will never
|
|
# contain. Naming them also means adding a member does not silently add a
|
|
# binary to the image.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS nestri-src
|
|
WORKDIR /build/nestri
|
|
COPY Cargo.toml Cargo.lock ./
|
|
COPY crates/nesprotocol crates/nesprotocol
|
|
COPY crates/nespyro crates/nespyro
|
|
# Not a guest component and not installed below — it runs on a stranger's own
|
|
# machine. It is here because `cargo` loads every workspace member's manifest
|
|
# before it builds anything, so a member missing from the context fails the
|
|
# build outright with `failed to read .../Cargo.toml`. Copying it costs a few
|
|
# files; the member list is the thing that decides, not this build.
|
|
COPY apps/nesdoctor apps/nesdoctor
|
|
COPY apps/nesinit apps/nesinit
|
|
COPY apps/nescope apps/nescope
|
|
COPY apps/neshub apps/neshub
|
|
COPY apps/neswire apps/neswire
|
|
COPY apps/nescapture apps/nescapture
|
|
COPY apps/nesgamepad apps/nesgamepad
|
|
|
|
FROM nestri-src AS nestri-build
|
|
RUN --mount=type=cache,target=/root/.cargo/registry \
|
|
--mount=type=cache,target=/build/nestri/target \
|
|
cargo build --release \
|
|
-p nesinit -p nescope -p neshub -p neswire -p nescapture -p nesgamepad && \
|
|
mkdir -p /artifacts/nestri/usr/bin /artifacts/nestri/usr/lib \
|
|
/artifacts/nestri/usr/share/vulkan/implicit_layer.d && \
|
|
install -Dm755 target/release/nesinit /artifacts/nestri/usr/bin/nesinit && \
|
|
install -Dm755 target/release/nescope /artifacts/nestri/usr/bin/nescope && \
|
|
install -Dm755 target/release/neshub /artifacts/nestri/usr/bin/neshub && \
|
|
install -Dm755 target/release/neswire /artifacts/nestri/usr/bin/neswire && \
|
|
install -Dm755 target/release/nesgamepad /artifacts/nestri/usr/bin/nesgamepad && \
|
|
install -Dm755 target/release/libnescapture_layer.so \
|
|
/artifacts/nestri/usr/lib/libnescapture_layer.so && \
|
|
install -Dm644 apps/nescapture/manifest/VK_LAYER_nescapture.json \
|
|
/artifacts/nestri/usr/share/vulkan/implicit_layer.d/VK_LAYER_nescapture.json && \
|
|
find /artifacts/nestri -type f -printf '/%P\n' > /artifacts/nestri/.manifest
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# virtio-nvgpu's Vulkan layer — for guests on an NVIDIA host
|
|
#
|
|
# On an NVIDIA host the guest runs NVIDIA's own Vulkan driver over a
|
|
# forwarded device, and that driver offers ray tracing, optical flow and a
|
|
# few others it then cannot create a device with: vkCreateDevice fails, and
|
|
# an engine that enables what it is offered does not start. The layer hides
|
|
# them. It acts only on an NVIDIA device in a guest with the virtio-nvgpu
|
|
# driver, so on an AMD or Intel box it is loaded and does nothing.
|
|
#
|
|
# Built from virtio-nvgpu's own tree at NVGPU_REF, the same ref the kernel
|
|
# build takes the guest driver from (build/Makefile), because the layer's job
|
|
# is to say what that driver and its backend can do. The commit is recorded
|
|
# in /usr/share/nestri/virtio-nvgpu.rev.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS nvgpu-layer-build
|
|
ARG NVGPU_GIT=https://github.com/nestrilabs/virtio-nvgpu.git
|
|
ARG NVGPU_REF=dev
|
|
RUN --mount=type=cache,target=/root/.cargo/registry \
|
|
git init -q /build/virtio-nvgpu && cd /build/virtio-nvgpu && \
|
|
git fetch -q --depth=1 "${NVGPU_GIT}" "${NVGPU_REF}" && git checkout -q FETCH_HEAD && \
|
|
cargo build --release -p nvgpu-vklayer && \
|
|
install -Dm755 target/release/libVkLayer_nvgpu.so \
|
|
/artifacts/nvgpu/usr/lib/libVkLayer_nvgpu.so && \
|
|
mkdir -p /artifacts/nvgpu/usr/share/vulkan/implicit_layer.d /artifacts/nvgpu/usr/share/nestri && \
|
|
sed 's|"./libVkLayer_nvgpu.so"|"libVkLayer_nvgpu.so"|' vklayer/VkLayer_nvgpu.json \
|
|
> /artifacts/nvgpu/usr/share/vulkan/implicit_layer.d/VkLayer_nvgpu.json && \
|
|
grep -q '"library_path": "libVkLayer_nvgpu.so"' \
|
|
/artifacts/nvgpu/usr/share/vulkan/implicit_layer.d/VkLayer_nvgpu.json && \
|
|
git rev-parse HEAD > /artifacts/nvgpu/usr/share/nestri/virtio-nvgpu.rev && \
|
|
cd / && rm -rf /build/virtio-nvgpu && \
|
|
find /artifacts/nvgpu -type f -printf '/%P\n' > /artifacts/nvgpu/.manifest
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# virtio-nvgpu's host half — shipped beside the guest's, taken out on the host
|
|
#
|
|
# `vhost-user-nvgpu` serves a box's forwarded GPU and `nvgpu-userspace`
|
|
# stages the host's NVIDIA driver for its guests. They run on the host, never
|
|
# in a guest: the host agent's onboarding moves /boot/host out of this image
|
|
# the way it moves the kernel, and turns forwarding on when it finds the
|
|
# NVIDIA driver loaded. In this image so they are always from the same
|
|
# NVGPU_REF as the guest driver and the layer above.
|
|
#
|
|
# Static (musl), because they run on whatever distribution the host is, and
|
|
# this one's glibc is newer than most of them.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS nvgpu-host-build
|
|
ARG NVGPU_GIT=https://github.com/nestrilabs/virtio-nvgpu.git
|
|
ARG NVGPU_REF=dev
|
|
RUN rustup target add x86_64-unknown-linux-musl
|
|
RUN --mount=type=cache,target=/root/.cargo/registry \
|
|
git init -q /build/virtio-nvgpu && cd /build/virtio-nvgpu && \
|
|
git fetch -q --depth=1 "${NVGPU_GIT}" "${NVGPU_REF}" && git checkout -q FETCH_HEAD && \
|
|
cargo build --release --target x86_64-unknown-linux-musl -p device --features vhost-user \
|
|
--bin vhost-user-nvgpu --bin nvgpu-userspace && \
|
|
for b in vhost-user-nvgpu nvgpu-userspace; do \
|
|
install -Dm755 "target/x86_64-unknown-linux-musl/release/$b" "/artifacts/nvgpu-host/boot/host/$b" && \
|
|
strip "/artifacts/nvgpu-host/boot/host/$b"; \
|
|
done && \
|
|
git rev-parse HEAD > /artifacts/nvgpu-host/boot/host/virtio-nvgpu.rev && \
|
|
cd / && rm -rf /build/virtio-nvgpu
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# The guest kernel — shipped inside the image it boots
|
|
#
|
|
# A host takes it back out as a file beside the root image when it unpacks
|
|
# this one (the host agent's onboarding), because the VMM loads a kernel from the host
|
|
# and never from the guest's disk. Shipping the two as one image is what
|
|
# keeps them a pair: the driver built into this kernel and the Vulkan layer
|
|
# above come from the same NVGPU_REF, and a host can only ever have the
|
|
# kernel that matches its root.
|
|
#
|
|
# The same script `make kernel` runs, on a fresh clone every time: an image
|
|
# build has no warm tree to reuse, and the layer cache already skips this
|
|
# stage while its pins and its inputs are unchanged.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS kernel-build
|
|
ARG KERNEL_GIT=https://github.com/CachyOS/linux.git
|
|
ARG KERNEL_REF=cachyos-7.2.6-1
|
|
ARG KERNEL_MARCH=x86-64-v3
|
|
ARG NVGPU_GIT=https://github.com/nestrilabs/virtio-nvgpu.git
|
|
ARG NVGPU_REF=dev
|
|
RUN pacman -S --noconfirm --needed lld bc cpio perl && pacman -Scc --noconfirm
|
|
COPY build/kernel /kernel-build/kernel
|
|
COPY build/scripts/kernel-build.sh /kernel-build/scripts/kernel-build.sh
|
|
RUN KERNEL_GIT="${KERNEL_GIT}" KERNEL_REF="${KERNEL_REF}" KERNEL_MARCH="${KERNEL_MARCH}" \
|
|
KERNEL_SRC=/build/kernel KERNEL_OUTPUT=/artifacts/kernel/boot/vmlinux \
|
|
NVGPU_GIT="${NVGPU_GIT}" NVGPU_REF="${NVGPU_REF}" NVGPU_WORK=/build/nvgpu \
|
|
bash /kernel-build/scripts/kernel-build.sh && \
|
|
rm -rf /build/kernel /build/nvgpu
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# umu-launcher — built here, not installed from the distribution
|
|
#
|
|
# `umu-run` launches every title that is not Steam's, applying the game fixes
|
|
# Proton carries outside Steam the way launchers such as Heroic do.
|
|
#
|
|
# The distribution's package is not used because of what it declares: it
|
|
# depends on a long list of `lib32-*` libraries and desktop packages that exist
|
|
# for the Steam Linux Runtime container, which this image never runs — Proton
|
|
# runs on the libraries in here (see the compat_check stage). Building from
|
|
# source costs the package and none of that.
|
|
#
|
|
# Built against the distribution's own `urllib3` and `pyzstd` rather than the
|
|
# copies it can vendor: vendoring builds both from source with pinned build
|
|
# tooling, and a rolling distribution moves past those pins (a
|
|
# `setuptools-scm` too new for them failed the build). The image installs the
|
|
# two as ordinary Python packages, with no 32-bit anything.
|
|
#
|
|
# Pinned by tag; it moves deliberately.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM builder AS umu-build
|
|
ARG UMU_GIT=https://github.com/Open-Wine-Components/umu-launcher.git
|
|
ARG UMU_TAG=1.4.4
|
|
RUN pacman -Syu --noconfirm --needed \
|
|
make scdoc gcc \
|
|
python-build python-hatchling python-installer python-pip \
|
|
&& pacman -Scc --noconfirm
|
|
RUN git clone --depth 1 --branch "${UMU_TAG}" --recurse-submodules --shallow-submodules \
|
|
"${UMU_GIT}" /umu && \
|
|
cd /umu && \
|
|
./configure.sh --prefix=/usr --use-system-pyzstd --use-system-urllib && \
|
|
make && \
|
|
make DESTDIR=/artifacts/umu install
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# os-base — the Arch rootfs itself
|
|
#
|
|
# `FROM archlinux:base` directly, and `pacman -S` as plain RUN steps — not a
|
|
# privileged host `chroot` into a hand-extracted tarball, which would need
|
|
# /proc, /sys and /dev bind-mounted in first (they don't exist inside a chroot
|
|
# target until something puts them there). A Containerfile RUN step already
|
|
# executes inside a real container with its own /proc, /sys, /dev, so there is
|
|
# no bind-mount step to write at all.
|
|
#
|
|
# # systemd goes; systemd-libs stays
|
|
#
|
|
# Nothing in the package list below depends on `systemd`, and two things in it
|
|
# — dbus-daemon and wireplumber — link `libsystemd.so.0`, which comes from the
|
|
# separate `systemd-libs` package. So the removal is `-Rdd` of `systemd` and
|
|
# `systemd-sysvcompat` only, which is normal rather than a compromise: keeping
|
|
# the library while having no PID 1 from it is exactly how a distribution
|
|
# without systemd ships these same programs.
|
|
#
|
|
# Removing the package also removes its pacman hooks, which is the point. The
|
|
# hooks call `systemd-tmpfiles`, `systemd-sysusers` and `udevadm`; leaving them
|
|
# behind while deleting what they call is how a later transaction fails
|
|
# obscurely, and a previous attempt at this image lost two services to exactly
|
|
# that.
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM docker.io/archlinux:base AS os-base
|
|
|
|
# The base image ships an unsigned local keyring, so upgrading
|
|
# `archlinux-keyring` runs a hook that reports `There is no secret key
|
|
# available to sign with` and then `error: command failed to execute
|
|
# correctly`. It is cosmetic and it is also every Arch container's build log.
|
|
# One line fixes it, and it is worth the seconds: a build that always prints an
|
|
# error is a build nobody reads an error out of.
|
|
RUN pacman-key --init
|
|
|
|
# Installed first and removed second, so every dependency resolves normally
|
|
# before anything is taken out from under it.
|
|
#
|
|
# Gone with the init system: `openrc`, `udev`, `dbus-openrc`. `udev` is not
|
|
# replaced by anything — `devtmpfs` creates the nodes and init sets the two
|
|
# modes that matter, because the compositor takes input through Wayland and
|
|
# opens nothing udev provides. ref(d-0064)
|
|
#
|
|
# `logrotate` is also gone, and that one is not about the init system: a box
|
|
# keeps no logs to rotate. What is worth reading leaves over the control
|
|
# channel, and `/var/log` is a small tmpfs that is discarded with the box.
|
|
# `mesa` is not in this list, and the two `--assume-installed` flags are why.
|
|
#
|
|
# The distribution's Mesa used to be installed so that every runtime dependency
|
|
# of *a* Mesa was present and correctly versioned, and ours was then overlaid
|
|
# on top. That worked for the unversioned filenames and not for the versioned
|
|
# one: `libgallium-<version>.so` from the package sat beside ours, 53 MB of it,
|
|
# referenced by nothing. Installing it to overwrite most of it was always the
|
|
# roundabout way round; telling pacman the dependency is already satisfied is
|
|
# the direct one.
|
|
#
|
|
# Exactly two flags are needed and both were checked by dropping each in turn:
|
|
# `mesa` is the name two packages depend on, and `opengl-driver` is a virtual
|
|
# provide `libglvnd` requires that only a real driver package satisfies. The
|
|
# other three names Mesa provides — `mesa-libgl`, `libva-driver`,
|
|
# `libva-mesa-driver` — change nothing here, so they are not listed.
|
|
#
|
|
# What makes this safe is that our Mesa is a superset for this image's
|
|
# purposes: it builds the drivers a box can actually use and the package's
|
|
# other ones (apple, asahi, armada, d3d12) are for hardware no box has. What it
|
|
# does *not* build is a software rasteriser, so there is no llvmpipe fallback —
|
|
# a box with no working GPU path now fails instead of rendering slowly, which
|
|
# is the honest outcome for something that exists to stream frames.
|
|
#
|
|
# Two packages below are explicit *because* Mesa is gone, and both used to
|
|
# arrive as its dependencies: `llvm-libs`, which the radeonsi driver links for
|
|
# shader compilation, and `lm_sensors`, which it links for `libsensors.so.5`.
|
|
# The second was found by the check further down rather than by reading the
|
|
# list — dropping a package takes its dependency tree with it, and the part of
|
|
# that tree something else was quietly using is not visible from here.
|
|
# Created before the install, with the numbers the distribution reserves for them,
|
|
# because the packages hand ownership of a few directories to these names and
|
|
# the hook that would have made the users goes with systemd. `polkit` is here
|
|
# for `rtkit`, which will not grant realtime priority to anything without it;
|
|
# see the services table in `nesinit`.
|
|
RUN groupadd -r -g 102 polkitd && useradd -r -u 102 -g polkitd -d / -s /usr/bin/nologin polkitd && \
|
|
groupadd -r -g 133 rtkit && useradd -r -u 133 -g rtkit -d / -s /usr/bin/nologin rtkit
|
|
|
|
RUN pacman -Syu --noconfirm --needed \
|
|
--assume-installed mesa --assume-installed opengl-driver \
|
|
dbus polkit rtkit \
|
|
iptables iproute2 \
|
|
libglvnd libdrm libepoxy libxxf86vm libinput wayland \
|
|
expat zlib llvm-libs lm_sensors elfutils libva shaderc vulkan-icd-loader \
|
|
pixman libxkbcommon xcb-util-keysyms xorg-xwayland \
|
|
pipewire pipewire-audio wireplumber opus \
|
|
python libunwind sdl2-compat \
|
|
python-xlib python-urllib3 python-pyzstd \
|
|
libvpx speex libtheora libvdpau jsoncpp \
|
|
strace lsof less \
|
|
&& rm -f /usr/share/libalpm/hooks/dbus-reload.hook \
|
|
&& pacman -Rdd --noconfirm systemd systemd-sysvcompat \
|
|
&& pacman -Scc --noconfirm
|
|
|
|
# `libvpx`, `speex`, `libtheora`, `libvdpau` and `jsoncpp` are not ours at all:
|
|
# they are what Proton's own GStreamer plugins link against, and Proton does
|
|
# not carry them. Without them `libgstlibav.so` and `libgstspeex.so` fail to
|
|
# load (`libvpx.so.12: cannot open shared object file`), so a game whose videos
|
|
# go through Wine's media stack plays them as nothing. Measured 2026-10-15 by
|
|
# listing every NEEDED library of the published Proton against this image;
|
|
# these five were the whole gap, for the stock build and the patched one
|
|
# alike. They track the sonames Proton was built against, so a Proton built
|
|
# on a newer Arch can need this list rebuilt with it.
|
|
#
|
|
# `strace`, `lsof` and `less` are for the operator's shell (`neslet box exec`):
|
|
# a root shell in a box that cannot trace a process, list what it has open or
|
|
# page through a log is the situation that feature exists to get out of. They
|
|
# are a few hundred kilobytes between them.
|
|
#
|
|
# `libunwind` is Wine's, not ours. `ntdll.so` links it, so without it every
|
|
# process Wine starts dies at `could not load ntdll.so`, which is the first
|
|
# thing it loads and reads like Wine itself being broken. Found 2026-09-12,
|
|
# after the prefix had already been created -- so the session got past every
|
|
# check that Proton was present and usable.
|
|
#
|
|
# `python` is not a build dependency here -- the builder stage has its own for
|
|
# Mesa -- it is a *runtime* one. The compatibility tool's entry point is a
|
|
# Python script, so a box without an interpreter starts a game and the launch
|
|
# ends with `env: 'python3': No such file or directory` and an exit status that
|
|
# reads like an ordinary finish. Found 2026-09-12, on the first session that
|
|
# got as far as launching one.
|
|
|
|
# `sdl2-compat` is Wine's controller support, and the only one there is for a
|
|
# controller that is not a Steam Input one. Proton's device bus reads evdev
|
|
# controllers through SDL2 and nothing else -- its udev backend defers every
|
|
# evdev device to the SDL one on purpose -- and it loads the library with
|
|
# `dlopen`, so its absence fails nothing at build time or at launch. A game
|
|
# simply sees no controller, while `nesgamepad` reports it plugged in. Found
|
|
# 2026-09-27, on the first session with one. Real Steam brings SDL in its
|
|
# runtime, which is why this is never missed anywhere else. On Arch the
|
|
# library is SDL2's API over SDL3, which arrives as its dependency.
|
|
|
|
# `dbus-reload.hook` is deleted above, before the removal rather than after,
|
|
# and it is the whole reason that line is there: the hook runs
|
|
# `/usr/share/libalpm/scripts/systemd-hook`, which systemd owns, so the
|
|
# transaction that removes systemd trips its own leftover on the way out —
|
|
# `call to execv failed`, then `error: command failed to execute correctly`.
|
|
# pacman treats a post-transaction hook failure as non-fatal, so the build
|
|
# survives it and the image is fine; what it leaves is an error message in
|
|
# every future transaction and a reader with no way to tell it from a real
|
|
# one. Removing the hook first means the error never happens.
|
|
|
|
# Nothing left may point at a program that is not here.
|
|
#
|
|
# The specific case above is fixed; this is the general one, and it exists
|
|
# because a hook calling a deleted binary is the exact shape of the failure
|
|
# that took two services off a previous version of this image. A build error
|
|
# is a much better place to find the next one than a log.
|
|
RUN for hook in /usr/share/libalpm/hooks/*.hook; do \
|
|
exec_line="$(awk -F'= *' '/^Exec/ { print $2; exit }' "$hook")"; \
|
|
program="${exec_line%% *}"; \
|
|
case "$program" in /*) ;; *) continue ;; esac; \
|
|
test -e "$program" \
|
|
|| { echo "$(basename "$hook") runs $program, which is not in the image" >&2; exit 1; }; \
|
|
done
|
|
|
|
# The check, because the removal above is the kind of thing a later `pacman
|
|
# -Syu` undoes quietly. A box with systemd's PID 1 back in it boots something
|
|
# other than `nesinit`, and the symptom is a guest that never dials out.
|
|
RUN test ! -e /usr/lib/systemd/systemd \
|
|
|| { echo "systemd's PID 1 is back in the image" >&2; exit 1; }
|
|
|
|
# `groupadd -f`, because some of these already exist in the base image and the
|
|
# rest have to. Nothing creates them at runtime any more: udev's rules did that
|
|
# for device nodes, and with udev gone init sets the two modes that matter
|
|
# directly. ref(d-0064)
|
|
#
|
|
# **Two users, and they must stay two.** `nestri` runs the services that come
|
|
# with this image; `nesplay` is who a workload runs as. Sharing one user between
|
|
# them is what lets workload code impersonate a service — it can replace the
|
|
# socket a service listens on and answer in its place, and the answer that
|
|
# matters is the address a client is told to connect to. Init refuses an address
|
|
# served by the workload's own user, so a single shared user does not merely
|
|
# weaken that check, it makes every session fail it.
|
|
#
|
|
# The uid a workload actually runs as is chosen by whoever asks for the box, not
|
|
# here; this account exists so that the number has a home, a shell and a name in
|
|
# `ps`, and so the separation has somewhere to be written down.
|
|
RUN groupadd -f audio && groupadd -f video && groupadd -f input && groupadd -f render && \
|
|
useradd -m -u 1000 -s /bin/bash nestri && \
|
|
for g in audio video input render; do gpasswd -a nestri "$g" >/dev/null; done && \
|
|
useradd -m -u 1001 -s /bin/bash nesplay && \
|
|
for g in audio video input render; do gpasswd -a nesplay "$g" >/dev/null; done
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# runtime — everything common to debug and prod
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM os-base AS runtime
|
|
|
|
# This is what GHCR actually uses to connect a pushed image back to its
|
|
# repo — not a setting to toggle after the fact, a label the image has to
|
|
# carry. Without it a manually-pushed image shows no "used by" repo on its
|
|
# package page even though this Containerfile is exactly what built it.
|
|
LABEL org.opencontainers.image.source="https://github.com/nestrilabs/nestri"
|
|
|
|
# Our own builds, overlaid on the distro's mesa. The distro package landed
|
|
# first (above) so every runtime dependency of *a* Mesa is present and
|
|
# correctly versioned; this overwrites its .so files with ours.
|
|
#
|
|
# COPY --from runs as root inside this build with no invoking-user uid to
|
|
# stamp onto / or /usr/bin, unlike a host-side `podman cp` + `cp -a` — so
|
|
# there is no ownership-sanity-check to write here. Nothing to catch, on
|
|
# purpose, not an oversight.
|
|
COPY --from=mesa-build /artifacts/mesa/.manifest /tmp/mesa.manifest
|
|
COPY --from=nestri-build /artifacts/nestri/.manifest /tmp/nestri.manifest
|
|
COPY --from=nvgpu-layer-build /artifacts/nvgpu/.manifest /tmp/nvgpu.manifest
|
|
RUN cat /tmp/mesa.manifest /tmp/nestri.manifest /tmp/nvgpu.manifest > /tmp/.strip-manifest && \
|
|
rm -f /tmp/mesa.manifest /tmp/nestri.manifest /tmp/nvgpu.manifest
|
|
COPY --from=mesa-build /artifacts/mesa /
|
|
COPY --from=nestri-build /artifacts/nestri /
|
|
COPY --from=nvgpu-layer-build /artifacts/nvgpu /
|
|
COPY --from=umu-build /artifacts/umu /
|
|
# The kernel this image is booted with, and the host's half of GPU
|
|
# forwarding, for the host to take out. See the kernel-build and
|
|
# nvgpu-host-build stages.
|
|
COPY --from=kernel-build /artifacts/kernel /
|
|
COPY --from=nvgpu-host-build /artifacts/nvgpu-host /
|
|
|
|
# **No Proton here.** It is attached to a box at run time, mounted at
|
|
# /nestri/compat, so this image has no Proton path for a launch to fall back to
|
|
# and a box started without one fails saying where it looked.
|
|
|
|
RUN ldconfig
|
|
|
|
COPY build/etc/ /etc/
|
|
|
|
# `nesinit` is PID 1, and `/usr/bin/init` is the fallback for a kernel started
|
|
# without an explicit `init=`. `systemd-sysvcompat` used to own that path and
|
|
# was removed with the rest of systemd, so nothing else claims it.
|
|
RUN ln -sf nesinit /usr/bin/init
|
|
|
|
# One id per boot, not one per image.
|
|
#
|
|
# `dbus-uuidgen --ensure=/etc/machine-id` used to run here, which baked one id
|
|
# into the image and made every box built from it the same machine. Init writes
|
|
# a fresh one to /run at boot instead, so both of these are symlinks into a
|
|
# tmpfs — which is also the only place they could be, with a read-only root.
|
|
RUN rm -f /etc/machine-id /var/lib/dbus/machine-id && \
|
|
mkdir -p /var/lib/dbus && \
|
|
ln -sf /run/machine-id /etc/machine-id && \
|
|
ln -sf /run/machine-id /var/lib/dbus/machine-id
|
|
|
|
# Session mount points. The guest root is read-only at runtime, so a runtime
|
|
# mkdir gets EROFS and takes a service down before it starts — these have to
|
|
# already exist in the image.
|
|
#
|
|
# `/nestri/logs` is a mount point and nothing mounts it from in here any more:
|
|
# a share arrives because the caller named it in the boot descriptor, which is
|
|
# the same rule every other share follows. The directory stays so that naming
|
|
# it works.
|
|
RUN mkdir -p /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \
|
|
chmod 0755 /nestri /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \
|
|
mkdir -p /dev/shm && chmod 1777 /dev/shm && \
|
|
mkdir -p /run/user/1000 /var/log && \
|
|
# The distribution's own empty `fstab` goes with ours. Nothing in a box
|
|
# reads either: init mounts what a box always needs, and every share comes
|
|
# from the boot descriptor. A file that looks like it configures mounts and
|
|
# is read by nothing is a file somebody edits expecting an effect.
|
|
rm -f /etc/network/interfaces /etc/inittab /etc/fstab
|
|
|
|
# Nothing in this image may be an init system except `nesinit`.
|
|
#
|
|
# A service manager arriving as a dependency of something innocuous is the
|
|
# failure this catches, and it is silent otherwise: the extra init does not run
|
|
# — the kernel is told which one to start — it just sits there with its own
|
|
# ideas about what the box should be doing, waiting for somebody to wire it in.
|
|
RUN for intruder in /usr/lib/systemd/systemd /sbin/openrc-init /usr/bin/openrc-init \
|
|
/sbin/runit-init /usr/bin/runit-init /sbin/dinit /usr/bin/dinit; do \
|
|
test ! -e "$intruder" || { echo "a second init is in the image: $intruder" >&2; exit 1; }; \
|
|
done
|
|
|
|
# What `nesinit` will look for at runtime, checked while there is somebody to
|
|
# read the failure.
|
|
#
|
|
# It is a table compiled into a binary, so a missing program is not a build
|
|
# error — it is a service that does not come up in a box somebody is waiting
|
|
# on, reported over the control channel and read hours later. Checking here
|
|
# turns that into a failed build.
|
|
# Everything this image promises must resolve the libraries it links.
|
|
#
|
|
# This is the check the Mesa change needs: dropping a package that provided
|
|
# libraries is how a binary ends up resolving nothing, and the symptom is not a
|
|
# build failure — it is a service that will not start in a box somebody is
|
|
# waiting on, or a render path that is missing rather than slow. It caught
|
|
# exactly that on the first run, and the missing library was two levels down a
|
|
# dependency tree nobody had reason to read.
|
|
#
|
|
# **Named rather than swept, and that is deliberate.** A sweep over everything
|
|
# in /usr/lib fails on a stock image: a distribution ships optional plugins
|
|
# whose optional dependencies are not installed — pinentry's Qt build, mpg123's
|
|
# JACK output, libdecor's GTK backend — and every one of those was already
|
|
# unresolved before this stage existed. A check that reports a dozen things
|
|
# nobody intends to load is a check the next person deletes. This list is what
|
|
# the image is *for*: the components, the services init starts, and the chain
|
|
# between a workload and the GPU. Wine's own core is checked by the
|
|
# `compat_check` stage, against the Proton it is actually going to run.
|
|
#
|
|
# **Wine was added after it was missed**, and then narrowed twice, which is
|
|
# worth recording so nobody widens it again.
|
|
#
|
|
# It was missed because the list covered everything this image ships *of ours*
|
|
# and nothing of the compatibility tool's, so an unresolved `libunwind.so.8`
|
|
# behind `ntdll.so` survived a build whose whole purpose is catching that, and
|
|
# surfaced as a session that created a prefix and could not start one process
|
|
# in it.
|
|
#
|
|
# The obvious fix -- sweep every `*-unix/*.so` -- is wrong in both directions.
|
|
# It is noisy: those objects are Wine's optional backends, and their
|
|
# dependencies are a camera library, a media stack, a VR loader, a smartcard
|
|
# daemon and OpenCL, none of which belong in a box. And it cannot see what it
|
|
# is checking: Wine's unix objects **link each other by soname** and are
|
|
# resolved by Wine's own loader rather than by `ld.so`, so `ldd` reports
|
|
# `ntdll.so` and `win32u.so` themselves as missing while they sit in the same
|
|
# directory. Forty files, every one a false positive, hiding the one real
|
|
# entry.
|
|
#
|
|
# So: the programs in `bin/`, which are ordinary ELF and resolve normally,
|
|
# `ntdll.so`, which is the first thing Wine loads and the one that linked the
|
|
# missing library, and `winepulse.so`, which is audio. That is exactly the
|
|
# failure that got through, with none of the noise. The Windows-side DLLs beside
|
|
# them are not ELF and `ldd` skips them anyway.
|
|
#
|
|
# Each file is resolved with its own directory on the library path. That is
|
|
# where Wine's loader finds the siblings a unix object links by soname, so
|
|
# `winepulse.so` naming `ntdll.so` resolves the way it does at run time, and
|
|
# what is left is a library the image really lacks. For every other file in
|
|
# the list it changes nothing.
|
|
# The output is one file per line with its own missing libraries under it, and
|
|
# then every missing library once at the end. That last list is what somebody
|
|
# acts on -- it is the set of packages to add -- and forty files each naming the
|
|
# same two libraries is not that list. An earlier version printed one
|
|
# comma-joined line and cut the wrong field out of `ldd`, so it named no
|
|
# libraries at all: `ldd` indents with a tab, which `tr -s ' '` does not
|
|
# collapse, so the second space-separated field is `=>`.
|
|
RUN failed=0; \
|
|
: > /tmp/missing-libs; \
|
|
for f in /usr/bin/nesinit /usr/bin/nescope /usr/bin/neshub /usr/bin/neswire \
|
|
/usr/bin/nesgamepad /usr/lib/libnescapture_layer.so /usr/lib/libSDL2-2.0.so.0 \
|
|
/usr/lib/libVkLayer_nvgpu.so \
|
|
/usr/bin/dbus-daemon /usr/bin/pipewire \
|
|
/usr/lib/polkit-1/polkitd /usr/lib/rtkit-daemon \
|
|
/usr/bin/wireplumber /usr/bin/ip \
|
|
/usr/lib/libgallium-*.so /usr/lib/libEGL_mesa.so.0 \
|
|
/usr/lib/libvulkan_*.so /usr/lib/dri/*.so /usr/lib/gbm/*.so; do \
|
|
[ -e "$f" ] || continue; \
|
|
libs="$(LD_LIBRARY_PATH="${f%/*}" ldd "$f" 2>/dev/null | awk '/not found/ { print $1 }')"; \
|
|
[ -n "$libs" ] || continue; \
|
|
failed=1; \
|
|
printf ' %s\n' "$f" >&2; \
|
|
printf ' %s\n' $libs >&2; \
|
|
printf '%s\n' $libs >> /tmp/missing-libs; \
|
|
done; \
|
|
if [ "$failed" != 0 ]; then \
|
|
echo "" >&2; \
|
|
echo " every library above, once each -- this is the list to install:" >&2; \
|
|
sort -u /tmp/missing-libs | sed 's/^/ /' >&2; \
|
|
exit 1; \
|
|
fi
|
|
|
|
RUN for required in /usr/bin/nesinit /usr/bin/nescope /usr/bin/neshub /usr/bin/neswire \
|
|
/usr/bin/nesgamepad /usr/lib/libSDL2-2.0.so.0 \
|
|
/usr/bin/dbus-daemon /usr/bin/pipewire \
|
|
/usr/lib/polkit-1/polkitd /usr/lib/rtkit-daemon \
|
|
/usr/bin/wireplumber /usr/bin/ip /usr/bin/python3 /usr/bin/umu-run; do \
|
|
test -x "$required" || { echo "the image is missing $required" >&2; exit 1; }; \
|
|
done
|
|
|
|
# `umu-run` has to start, not only exist. It is a Python program built in another
|
|
# stage: a module it imports that this image lacks, or an interpreter that moved
|
|
# a minor version between that stage and this one, is a game that never starts —
|
|
# and the launch reports a failure to run a script, not a missing module.
|
|
RUN umu-run --help >/dev/null 2>&1 \
|
|
|| { echo "umu-run does not start; run it by hand to see the module or interpreter it lacks" >&2; \
|
|
umu-run --help >&2 || true; exit 1; }
|
|
|
|
# No 32-bit libraries came with it. The distribution's umu package would have
|
|
# brought a long list of them for the container runtime this image never runs,
|
|
# which is why it is built instead of installed; this is the check that stays
|
|
# true if somebody swaps it back.
|
|
RUN ! pacman -Qq | grep -q '^lib32-' \
|
|
|| { echo "a lib32 package is installed:" >&2; pacman -Qq | grep '^lib32-' >&2; exit 1; }
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# runtime_prod — the default: `make build`
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM runtime AS runtime_prod
|
|
|
|
# No console is offered by either flavour: `nesinit` spawns no getty, because
|
|
# the way into a guest that will not boot is `init=/bin/bash` on the kernel
|
|
# command line, which needs nothing from the image but a shell. So the locked
|
|
# root account is belt and braces rather than the only thing standing between
|
|
# a box and a login prompt.
|
|
RUN passwd -l root
|
|
|
|
RUN while IFS= read -r f; do \
|
|
[ -f "$f" ] && strip --strip-unneeded "$f" 2>/dev/null || true; \
|
|
done < /tmp/.strip-manifest; \
|
|
rm -rf /tmp/.strip-manifest /var/cache/pacman/pkg/* /tmp/* /root/.cache \
|
|
/usr/share/man /usr/share/doc /usr/share/locale \
|
|
/usr/lib/cmake /usr/lib/pkgconfig /usr/share/pkgconfig /usr/include \
|
|
/usr/share/gir-1.0 /usr/lib/udev; \
|
|
find /usr/lib -name '*.a' -delete
|
|
|
|
RUN echo "NESTRI_STAGE=runtime_prod" >> /etc/os-release
|
|
|
|
|
|
# ───────────────────────────────────────────────────────────
|
|
# runtime_debug — `make build-debug`
|
|
# ───────────────────────────────────────────────────────────
|
|
FROM runtime AS runtime_debug
|
|
|
|
# `-Sy` and not `-Syu`: a full upgrade here can pull a package back in as
|
|
# somebody's dependency, and the one that matters is systemd. The check below
|
|
# catches it either way, but a debug image that fails to build is worse than
|
|
# one that is a few days behind on versions it only uses for `vulkaninfo`.
|
|
RUN pacman -Sy --noconfirm --needed vulkan-tools mesa-utils libva-utils && \
|
|
pacman -Scc --noconfirm
|
|
|
|
# The same guard as the runtime stage, because the transaction above is exactly
|
|
# the kind that quietly reinstates an init system.
|
|
RUN test ! -e /usr/lib/systemd/systemd \
|
|
|| { echo "systemd's PID 1 came back with the debug tools" >&2; exit 1; }
|
|
|
|
# Root has a password here and nothing offers a login prompt to type it into.
|
|
# It is for `su` from an `init=/bin/bash` shell, which is the whole debug route.
|
|
RUN echo 'root:nestri' | chpasswd
|
|
|
|
RUN echo "NESTRI_STAGE=runtime_debug" >> /etc/os-release
|