From 270304bca517270be7ce403df0f7f14c107e4c2b Mon Sep 17 00:00:00 2001 From: "KAAL1 (Bingus)" Date: Tue, 1 Sep 2026 15:05:48 +0300 Subject: [PATCH] feat(build): borealis-style multi-stage rootfs for the open guest components (#309) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What Adds `build/` — a Dockerfile with `mesa-build`, `nestri-build`, `os-base`, `runtime`, `runtime_prod` and `runtime_debug` stages, plus the `etc/` overlay, `mkimage.sh` and a `Makefile` — laid out the way [borealis](https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/main/project-borealis) lays out its own `build/`. This moves the guest rootfs formula into this repo, targeting the four open guest components already here: `nescope`, `neshub`, `neswire`, `nescapture`. ## Two structural properties worth calling out - **No privileged host chroot.** A bare `chroot` into a hand-extracted rootfs needs `/proc`, `/sys`, `/dev` bind-mounted in first. `os-base` here is `FROM artixlinux/artixlinux:base-openrc` directly with `pacman -S` as plain `RUN` steps — a Docker build step already has its own `/proc`/`/sys`/`/dev`. - **No host-side ownership bug to guard against.** `COPY --from=` runs as root inside the build with no invoking-user uid in the loop. ## Scope boundary **Deliberately excludes Proton and Valve's `steamclient.so`** — both closed, and `CLAUDE.md` forbids closed content in this repo. `runtime_prod`, tagged `nestrilabs/nestri:base`, is a complete, bootable, Steam-less image — and also the shared foundation other builds start from. Whatever layers Proton/Steam on top of it is a closed build outside this repo, by design. ## Known gap Nothing starts a payload yet — `nesinit` isn't open code — so `/etc/init.d/nescope` boots it in plain-compositor mode (no command after `--`) rather than running a game. Real and testable, just not a full session yet. Details in `build/README.md`. ## Status Built and tagged locally as `nestrilabs/nestri:base` (podman, no `--no-cache` issues, greptile's three findings all fixed and verified against a real build). Not yet packed into a disk image or run inside nesbox. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Greptile Summary

The PR adds a multi-stage Artix/OpenRC guest-rootfs build for the open Nestri components, with production and debug image flavors. - Builds patched Mesa and the Rust workspace in dedicated builder stages. - Assembles and configures the bootable guest environment and OpenRC services. - Packs a selected container image into an ext4 root filesystem while retaining rootless container storage access.

Confidence Score: 5/5

The PR appears safe to merge. No blocking failure remains.

Important Files Changed

| Filename | Overview | |----------|----------| | build/Dockerfile | Defines the complete multi-stage build, overlays repository-root-relative configuration paths, and creates production and debug runtime targets. | | build/Makefile | Provides consistent image build and packing targets using a repository-root context and matching image tags. | | build/scripts/mkimage.sh | Keeps container-runtime operations in the invoking user's storage while escalating only filesystem creation and mounting operations. | | build/etc/conf.d/nestri-user-env | Supplies the shared service environment and export function required by the OpenRC service scripts. | | build/etc/init.d/guest-net | Configures optional guest networking from kernel parameters or stable defaults. | | build/etc/init.d/neswire | Starts the audio sink after its dependencies and pins it as the PipeWire default once the graph is ready. |

Flowchart

```mermaid %%{init: {'theme': 'neutral'}}%% flowchart TD A[Arch builder] --> B[Mesa build] A --> C[Nestri workspace build] D[Artix OpenRC base] --> E[Common runtime] B --> E C --> E F[build/etc overlay] --> E E --> G[runtime_prod] E --> H[runtime_debug] G --> I[Container image] H --> J[Debug container image] I --> K[mkimage.sh] J --> K K --> L[ext4 rootfs] ``` Reviews (6): Last reviewed commit: ["refactor(build): rename the published im..."](https://github.com/nestrilabs/nestri/commit/6fecc8cc31bc425db952193074d4b9b0f861de89) | [Re-trigger Greptile](https://app.greptile.com/api/retrigger?id=58981739) --- .dockerignore | 18 ++ .gitignore | 9 +- build/Dockerfile | 295 ++++++++++++++++++ build/Makefile | 66 ++++ build/README.md | 90 ++++++ build/etc/conf.d/agetty.hvc0.debug | 3 + build/etc/conf.d/agetty.hvc0.prod | 3 + build/etc/conf.d/nescope | 16 + build/etc/conf.d/neshub | 18 ++ build/etc/conf.d/nestri-user-env | 30 ++ build/etc/conf.d/neswire | 32 ++ build/etc/fstab | 29 ++ build/etc/init.d/dbus-session | 33 ++ build/etc/init.d/dbus-system | 19 ++ build/etc/init.d/guest-net | 92 ++++++ build/etc/init.d/nescope | 33 ++ build/etc/init.d/neshub | 39 +++ build/etc/init.d/neswire | 93 ++++++ build/etc/init.d/pipewire | 45 +++ build/etc/init.d/wireplumber | 20 ++ build/etc/init.d/xdg-runtime | 44 +++ build/etc/profile.d/nestri-env.sh | 27 ++ build/etc/rc.conf | 27 ++ build/etc/resolv.conf | 2 + .../wireplumber.conf.d/50-nestri.conf | 41 +++ build/scripts/mkimage.sh | 67 ++++ 26 files changed, 1190 insertions(+), 1 deletion(-) create mode 100644 .dockerignore create mode 100644 build/Dockerfile create mode 100644 build/Makefile create mode 100644 build/README.md create mode 100644 build/etc/conf.d/agetty.hvc0.debug create mode 100644 build/etc/conf.d/agetty.hvc0.prod create mode 100644 build/etc/conf.d/nescope create mode 100644 build/etc/conf.d/neshub create mode 100644 build/etc/conf.d/nestri-user-env create mode 100644 build/etc/conf.d/neswire create mode 100644 build/etc/fstab create mode 100644 build/etc/init.d/dbus-session create mode 100644 build/etc/init.d/dbus-system create mode 100644 build/etc/init.d/guest-net create mode 100644 build/etc/init.d/nescope create mode 100644 build/etc/init.d/neshub create mode 100644 build/etc/init.d/neswire create mode 100644 build/etc/init.d/pipewire create mode 100644 build/etc/init.d/wireplumber create mode 100644 build/etc/init.d/xdg-runtime create mode 100644 build/etc/profile.d/nestri-env.sh create mode 100644 build/etc/rc.conf create mode 100644 build/etc/resolv.conf create mode 100644 build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf create mode 100644 build/scripts/mkimage.sh diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 00000000..d4a95622 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,18 @@ +# build/Dockerfile's context is the repo root (see build/Makefile), so this +# keeps the daemon-side context transfer to what it actually COPYs: the +# workspace manifests plus the five Rust members. Everything else here is +# TypeScript/tooling the guest rootfs build never touches. +.git +node_modules +target +build/output +docs +apps/api +apps/auth +packages +*.md +deno.lock +bun.lock +.env* +.zed +.github diff --git a/.gitignore b/.gitignore index 43ba845a..b8170f4f 100644 --- a/.gitignore +++ b/.gitignore @@ -6,12 +6,19 @@ node_modules .netlify .wrangler .svelte-kit -build +# JS-framework build-output dirs, at any depth — but not the top-level +# build/ directory, which is the guest rootfs build (see build/README.md), +# a real source tree we want tracked. +**/build +!/build # Rust /target **/*.rs.bk +# build/'s own output — the packed rootfs images, not source +/build/output/ + # OS .DS_Store Thumbs.db diff --git a/build/Dockerfile b/build/Dockerfile new file mode 100644 index 00000000..29abc082 --- /dev/null +++ b/build/Dockerfile @@ -0,0 +1,295 @@ +# ═══════════════════════════════════════════════════════════ +# nestri guest rootfs — the open half +# +# Builds a bootable Artix/OpenRC image containing Mesa (virtio-gpu native +# context) and the four open guest components: nescope, neshub, neswire, +# nescapture. Two leaf targets, selected with `--target`: +# +# runtime_prod stripped, root locked (default: `make build`) +# runtime_debug debug tools, autologin root (`make build-debug`) +# +# What is deliberately NOT here: Proton, Valve's steamclient.so, or anything +# else closed. nestri/CLAUDE.md is explicit that nothing closed enters this +# repo. Whatever layers those on top of runtime_prod is a closed build +# outside this repo — see build/README.md. +# +# Build from the repo root, not from build/: +# docker build -f build/Dockerfile --target runtime_prod -t nestri-guest . +# (`make build` in this directory does exactly that.) +# ═══════════════════════════════════════════════════════════ + + +# ─────────────────────────────────────────────────────────── +# initial / builder — Arch, not Artix +# +# The guest is Artix, but Artix's repos are Arch-derived and the toolchain/ +# glibc generation is the same, while the archlinux image is the +# better-maintained of the two for actually compiling things. Only build +# artifacts leave these stages. +# ─────────────────────────────────────────────────────────── +FROM docker.io/archlinux:base-devel AS initial +RUN pacman -Syu --noconfirm + +FROM initial AS builder +RUN pacman -S --noconfirm --needed \ + cmake meson ninja git pkgconf \ + python python-mako python-yaml python-packaging python-ply \ + bison flex \ + libpciaccess libepoxy libglvnd \ + libx11 libxext libxrandr libxshmfence libxfixes libxxf86vm libxcb \ + xcb-util-keysyms xorgproto \ + wayland wayland-protocols \ + expat zlib zstd libxml2 lm_sensors \ + llvm clang libclc spirv-tools spirv-llvm-translator glslang \ + elfutils libva libdrm directx-headers \ + rust rust-bindgen cbindgen \ + curl openssl \ + pixman libxkbcommon \ + vulkan-headers vulkan-icd-loader \ + pipewire shaderc opus \ + libinput \ + && pacman -Scc --noconfirm +WORKDIR /build +ENV ARTIFACTS=/artifacts + + +# ─────────────────────────────────────────────────────────── +# Mesa — the only piece still fetched from outside this tree +# ─────────────────────────────────────────────────────────── +FROM builder AS mesa-build + +ARG MESA_GIT=https://gitlab.freedesktop.org/mesa/mesa.git +ARG MESA_COMMIT=b78fc73dd898a7dfa87448a4b5ff4459a870e21a + +RUN git clone --depth=1 --revision="${MESA_COMMIT}" "${MESA_GIT}" /build/mesa-src && \ + cd /build/mesa-src && \ + meson setup builddir \ + -Dprefix=/usr \ + -Dbuildtype=release \ + -Dplatforms=wayland,x11 \ + -Dgallium-drivers=zink,radeonsi,iris \ + -Dvulkan-drivers=amd,intel \ + -Damdgpu-virtio=true \ + -Dintel-virtio-experimental=true \ + -Dvideo-codecs=all \ + -Degl=enabled \ + -Dglx=dri \ + -Dgles1=enabled \ + -Dgles2=enabled \ + -Dgbm=enabled \ + -Dgallium-va=enabled \ + -Db_ndebug=true && \ + ninja -C builddir && \ + DESTDIR=/artifacts/mesa ninja -C builddir install && \ + rm -rf /build/mesa-src && \ + find /artifacts/mesa -type f -printf '/%P\n' > /artifacts/mesa/.manifest + + +# ─────────────────────────────────────────────────────────── +# nestri workspace — same repo now, so this is COPY, not a private clone +# +# One `cargo build --release --workspace` rather than one stage per binary: +# that per-repo splitting existed because nescope/neswire/nescapture/the hub +# were four separate private repos and a stage boundary was the only way to +# stop bumping one from invalidating the others' build cache. They are one +# Cargo workspace with one Cargo.lock now, so a BuildKit cache mount on +# target/ gives the same isolation — cargo's own incremental compiler +# already knows nescope changing does not touch nesprotocol's .rlib — without +# four copies of every shared dependency getting compiled once per stage. +# ─────────────────────────────────────────────────────────── +FROM builder AS nestri-src +WORKDIR /build/nestri +COPY Cargo.toml Cargo.lock ./ +COPY crates/nesprotocol crates/nesprotocol +COPY apps/nescope apps/nescope +COPY apps/neshub apps/neshub +COPY apps/neswire apps/neswire +COPY apps/nescapture apps/nescapture + +FROM nestri-src AS nestri-build +RUN --mount=type=cache,target=/root/.cargo/registry \ + --mount=type=cache,target=/build/nestri/target \ + cargo build --release --workspace && \ + mkdir -p /artifacts/nestri/usr/bin /artifacts/nestri/usr/lib \ + /artifacts/nestri/usr/share/vulkan/implicit_layer.d && \ + install -Dm755 target/release/nescope /artifacts/nestri/usr/bin/nescope && \ + install -Dm755 target/release/neshub /artifacts/nestri/usr/bin/neshub && \ + install -Dm755 target/release/neswire /artifacts/nestri/usr/bin/neswire && \ + install -Dm755 target/release/libnescapture_layer.so \ + /artifacts/nestri/usr/lib/libnescapture_layer.so && \ + install -Dm644 apps/nescapture/manifest/VK_LAYER_nescapture.json \ + /artifacts/nestri/usr/share/vulkan/implicit_layer.d/VK_LAYER_nescapture.json && \ + find /artifacts/nestri -type f -printf '/%P\n' > /artifacts/nestri/.manifest + + +# ─────────────────────────────────────────────────────────── +# os-base — the Artix rootfs itself +# +# `FROM artixlinux/artixlinux:base-openrc` directly, and `pacman -S` as plain +# RUN steps — not a privileged host `chroot` into a hand-extracted tarball, +# which would need /proc, /sys and /dev bind-mounted in first (they don't +# exist inside a chroot target until something puts them there). A +# Dockerfile RUN step already executes inside a real container with its own +# /proc, /sys, /dev, so there is no bind-mount step to write at all. +# ─────────────────────────────────────────────────────────── +FROM docker.io/artixlinux/artixlinux:base-openrc AS os-base + +RUN pacman -Syu --noconfirm --needed \ + base openrc udev dbus dbus-openrc \ + iptables iproute2 \ + mesa libglvnd libdrm libepoxy libxxf86vm libinput wayland \ + expat zlib llvm-libs elfutils libva shaderc vulkan-icd-loader \ + pixman libxkbcommon xcb-util-keysyms xorg-xwayland \ + pipewire pipewire-audio wireplumber dbus logrotate opus \ + && pacman -Scc --noconfirm + +# groupadd -f so this is idempotent whether or not udev's rules already +# created these. +RUN groupadd -f audio && groupadd -f video && groupadd -f input && groupadd -f render && \ + useradd -m -u 1000 -s /bin/bash nestri && \ + for g in audio video input render; do gpasswd -a nestri "$g" >/dev/null; done + + +# ─────────────────────────────────────────────────────────── +# runtime — everything common to debug and prod +# ─────────────────────────────────────────────────────────── +FROM os-base AS runtime + +# This is what GHCR actually uses to connect a pushed image back to its +# repo — not a setting to toggle after the fact, a label the image has to +# carry. Without it a manually-pushed image shows no "used by" repo on its +# package page even though this Dockerfile is exactly what built it. +LABEL org.opencontainers.image.source="https://github.com/nestrilabs/nestri" + +# Our own builds, overlaid on the distro's mesa. The distro package landed +# first (above) so every runtime dependency of *a* Mesa is present and +# correctly versioned; this overwrites its .so files with ours. +# +# COPY --from runs as root inside this build with no invoking-user uid to +# stamp onto / or /usr/bin, unlike a host-side `podman cp` + `cp -a` — so +# there is no ownership-sanity-check to write here. Nothing to catch, on +# purpose, not an oversight. +COPY --from=mesa-build /artifacts/mesa/.manifest /tmp/mesa.manifest +COPY --from=nestri-build /artifacts/nestri/.manifest /tmp/nestri.manifest +RUN cat /tmp/mesa.manifest /tmp/nestri.manifest > /tmp/.strip-manifest && \ + rm -f /tmp/mesa.manifest /tmp/nestri.manifest +COPY --from=mesa-build /artifacts/mesa / +COPY --from=nestri-build /artifacts/nestri / +RUN ldconfig + +COPY build/etc/ /etc/ +RUN chmod +x /etc/init.d/* + +# dbus-session, pipewire and wireplumber carry no conf.d of their own — they +# just want the shared environment, so their conf.d is a symlink to it rather +# than a copy. nescope, neshub and neswire are deliberately absent from this +# loop: each has a conf.d file of its own (already landed by the COPY above) +# that sources nestri-user-env, because each needs settings the shared file +# does not carry. Symlinking them here would overwrite those. +RUN for svc in dbus-session pipewire wireplumber; do \ + ln -sf nestri-user-env "/etc/conf.d/${svc}"; \ + done + +RUN echo nesbox > /etc/hostname && dbus-uuidgen --ensure=/etc/machine-id + +# Session mount points. The guest root is read-only at runtime, so a runtime +# mkdir gets EROFS and takes a service down before it starts — these have to +# already exist in the image. +RUN mkdir -p /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \ + chmod 0755 /nestri /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \ + mkdir -p /dev/shm && chmod 1777 /dev/shm && \ + rm -f /etc/network/interfaces + +# Serial console: the only way into a guest that will not boot. +RUN rm -f /etc/inittab && \ + ln -sf agetty /etc/init.d/agetty.hvc0 && \ + { grep -qx hvc0 /etc/securetty || echo hvc0 >> /etc/securetty; } + +# Every init.d script calling nestri_export_env needs a conf.d that actually +# defines it. Worth a build-time check: the failure at runtime is nearly +# invisible — OpenRC sources the script, the undefined function is a +# "command not found" on stderr, sourcing still returns 0, and the service +# starts anyway with HOME and every XDG_* unset. +RUN missing=""; \ + for svc_script in /etc/init.d/*; do \ + grep -qE '^[[:space:]]*nestri_export_env\b' "$svc_script" 2>/dev/null || continue; \ + svc="$(basename "$svc_script")"; \ + . "/etc/conf.d/$svc" >/dev/null 2>&1; \ + command -v nestri_export_env >/dev/null 2>&1 || missing="$missing $svc"; \ + done; \ + [ -z "$missing" ] || { echo "init scripts call nestri_export_env with no conf.d providing it:$missing" >&2; exit 1; } + +# ── OpenRC service registration ── +# sysinit +RUN rc-update add devfs sysinit && \ + rc-update add dmesg sysinit && \ + rc-update add udev sysinit && \ + rc-update add udev-trigger sysinit && \ + (rc-update del kmod-static-nodes sysinit || true) + +# boot — guest-net replaces the distro's networking scripts entirely; it +# declares `provide net` and brings up lo itself. +RUN (rc-update del networking boot || true) && \ + (rc-update del network-async boot || true) && rm -f /etc/init.d/network-async && \ + rc-update add guest-net boot && \ + (rc-update del net.lo boot || true) && \ + (rc-update del netmount boot || true) && \ + (rc-update del netmount default || true) && \ + rc-update add hostname boot && \ + rc-update add xdg-runtime boot && \ + rc-update add cgroups boot && \ + (rc-update del syslog boot || true) + +# Boot-runlevel services with nothing to do in a freshly-built microVM: no +# physical console, no swap, and fsck would be checking a filesystem nobody +# has touched. +RUN for svc in fsck keymaps save-keymaps termencoding save-termencoding swap binfmt seedrng hwclock swclock; do \ + (rc-update del "$svc" boot || true); \ + (rc-update del "$svc" shutdown || true); \ + done + +# default — the user-facing stack. nescope is registered in plain-compositor +# mode (no payload after `--`): it comes up and waits for something to +# connect. Actually starting a payload is nesinit's job, and nesinit is not +# open code yet — see build/README.md. +RUN rc-update add agetty.hvc0 default && \ + for n in 1 2 3 4 5 6; do (rc-update del "agetty.tty${n}" default || true); done && \ + rc-update add dbus default && \ + rc-update add dbus-session default && \ + rc-update add pipewire default && \ + rc-update add wireplumber default && \ + rc-update add neshub default && \ + rc-update add neswire default && \ + rc-update add nescope default && \ + rm -f /etc/init.d/shared-root && (rc-update del shared-root boot || true) + + +# ─────────────────────────────────────────────────────────── +# runtime_prod — the default: `make build` +# ─────────────────────────────────────────────────────────── +FROM runtime AS runtime_prod + +RUN passwd -l root +COPY build/etc/conf.d/agetty.hvc0.prod /etc/conf.d/agetty.hvc0 + +RUN while IFS= read -r f; do \ + [ -f "$f" ] && strip --strip-unneeded "$f" 2>/dev/null || true; \ + done < /tmp/.strip-manifest; \ + rm -rf /tmp/.strip-manifest /var/cache/pacman/pkg/* /tmp/* /root/.cache \ + /usr/share/man /usr/share/doc /usr/share/locale \ + /usr/lib/cmake /usr/lib/pkgconfig /usr/share/pkgconfig /usr/include + +RUN echo "NESTRI_STAGE=runtime_prod" >> /etc/os-release + + +# ─────────────────────────────────────────────────────────── +# runtime_debug — `make build-debug` +# ─────────────────────────────────────────────────────────── +FROM runtime AS runtime_debug + +RUN pacman -Syu --noconfirm --needed vulkan-tools mesa-utils libva-utils && \ + pacman -Scc --noconfirm +RUN echo 'root:nestri' | chpasswd +COPY build/etc/conf.d/agetty.hvc0.debug /etc/conf.d/agetty.hvc0 + +RUN echo "NESTRI_STAGE=runtime_debug" >> /etc/os-release diff --git a/build/Makefile b/build/Makefile new file mode 100644 index 00000000..dc2c331b --- /dev/null +++ b/build/Makefile @@ -0,0 +1,66 @@ +SHELL := /bin/bash +.PHONY: build build-debug image image-debug clean help + +CONTAINER_RT := $(shell command -v docker 2>/dev/null || command -v podman 2>/dev/null) +ifeq ($(CONTAINER_RT),) +$(error "Neither docker nor podman found in PATH") +endif + +# The Dockerfile COPYs from apps/ and crates/, so the build context is the +# repo root, not this directory — same reason borealis's build/ *is* its own +# context: here the guest source lives one level up instead of inside build/. +CONTEXT := .. +# `ghcr.io/nestrilabs/nestri/base` (a package name with a `/` in it, matching +# how every other image this org has published is named) is a deliberate +# name, not just a tag: this is the image other builds start FROM — +# nesbox's jailer image extracts Mesa/virgl from it to keep the guest and +# host sides of the virtio-gpu native-context protocol on the same patched +# Mesa, and closed downstream builds layer Proton/Steam on top of it +# elsewhere. `runtime_prod`/`runtime_debug` already are exactly that: distro +# packages plus our own Mesa/nestri artifacts overlaid on top, nothing +# stripped that a downstream COPY --from= would miss — no separate +# unstripped tag is needed for this. +# +# The registry host is part of the name on purpose, including for local +# builds: a bare `nestrilabs/nestri` has no host, so anyone who pulls +# instead of building locally resolves it against Docker Hub by default, +# where it does not exist. One name, always pullable, beats a local-only +# short name and a different published one. +IMAGE_NAME := ghcr.io/nestrilabs/nestri/base +OUTPUT_DIR := output +ROOTFS_SIZE ?= 5G +FORCE_REBUILD ?= + +build: + DOCKER_BUILDKIT=1 $(CONTAINER_RT) build $(if $(FORCE_REBUILD),--no-cache,) \ + -f Dockerfile -t $(IMAGE_NAME):latest --target runtime_prod $(CONTEXT) + +build-debug: + DOCKER_BUILDKIT=1 $(CONTAINER_RT) build $(if $(FORCE_REBUILD),--no-cache,) \ + -f Dockerfile -t $(IMAGE_NAME):debug --target runtime_debug $(CONTEXT) + +# Not `sudo make image`/sudo'd in here: mkimage.sh runs as you and escalates +# only the specific commands that need root. Under rootless Podman, `make +# build` stores the image in *your* storage — sudo-ing the whole script would +# have root's podman look for that tag in its own, separate storage and fail +# to find it. +image: build + @mkdir -p $(OUTPUT_DIR) + bash scripts/mkimage.sh $(IMAGE_NAME):latest $(OUTPUT_DIR)/rootfs.ext4 $(ROOTFS_SIZE) + +image-debug: build-debug + @mkdir -p $(OUTPUT_DIR) + bash scripts/mkimage.sh $(IMAGE_NAME):debug $(OUTPUT_DIR)/rootfs-debug.ext4 $(ROOTFS_SIZE) + +clean: + rm -rf $(OUTPUT_DIR) + +help: + @echo "Usage:" + @echo " make build Build the runtime_prod container image" + @echo " make build-debug Build the runtime_debug container image" + @echo " make image Build + pack runtime_prod into output/rootfs.ext4" + @echo " make image-debug Build + pack runtime_debug into output/rootfs-debug.ext4" + @echo " make clean Remove output/" + @echo " make FORCE_REBUILD=1 ... Rebuild from scratch, no layer cache" + @echo " make ROOTFS_SIZE=8G image Override the packed image size (default 5G)" diff --git a/build/README.md b/build/README.md new file mode 100644 index 00000000..c43bedd0 --- /dev/null +++ b/build/README.md @@ -0,0 +1,90 @@ +# build/ — the guest rootfs + +Builds a bootable Artix/OpenRC image for the box's virtio-blk root: Mesa +(virtio-gpu native context) plus the four open guest components — +[`nescope`](../apps/nescope), [`neshub`](../apps/neshub), +[`neswire`](../apps/neswire), [`nescapture`](../apps/nescapture) — laid out +the way [borealis](https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/main/project-borealis) +lays out its `build/`: one big multi-stage `Dockerfile`, `--target` picks the +flavor, `etc/` holds the files that get overlaid onto the image verbatim. + +``` +build/ +├── Dockerfile everything, in stages: mesa-build, nestri-build, +│ os-base, runtime, runtime_prod, runtime_debug +├── etc/ overlaid onto the image's /etc as-is +├── scripts/ +│ └── mkimage.sh docker export → raw ext4, for nesbox's virtio-blk +├── Makefile +└── output/ `make image` writes here (gitignored) +``` + +```sh +make build # docker build --target runtime_prod → ghcr.io/nestrilabs/nestri/base:latest +make build-debug # docker build --target runtime_debug → ghcr.io/nestrilabs/nestri/base:debug +make image # + pack into output/rootfs.ext4 +make image-debug # + pack into output/rootfs-debug.ext4 +``` + +## Design notes + +Three things worth knowing about how this is put together: + +1. **No privileged host chroot.** A bare `chroot` into a hand-extracted + rootfs needs `/proc`, `/sys`, `/dev` bind-mounted in first — they don't + exist inside a chroot target until something puts them there. `os-base` + here is `FROM artixlinux/artixlinux:base-openrc` directly, with + `pacman -S` as plain `RUN` steps — a Docker build step already runs + inside a real container with its own `/proc`, `/sys`, `/dev`, so that + whole bind-mount mechanism has nothing to do. + +2. **No host-side ownership bug to guard against.** `COPY --from=` runs as + root inside the build with no host user in the loop, so there's no + invoking-user uid getting stamped onto `/`, `/usr/bin`, or anywhere else + a build step touches — a failure mode some overlay approaches need an + explicit sanity check for doesn't exist here to check for. + +3. **One `cargo build --release --workspace`, not one stage per binary.** + `nescope`, `neshub`, `neswire` and `nescapture` share one Cargo workspace + and one `Cargo.lock` — a BuildKit cache mount on `target/` gives cargo's + own incremental compiler per-crate isolation without needing a separate + Docker stage (and a separate full rebuild of `nesprotocol`) per binary. + +**What is deliberately not here: Proton, and Valve's `steamclient.so`.** +`nestri/CLAUDE.md` is explicit — *"Nothing closed may enter this repo. Not +source, not a dependency, not a directory that 'looked convenient'."* Both +are closed. `runtime_prod` from this Dockerfile — tagged +`ghcr.io/nestrilabs/nestri/base:latest` — is a complete, bootable, Steam-less guest image, +and also the shared foundation other builds start from: nesbox's jailer +image (see `nesbox/build/`) extracts Mesa and virglrenderer from it so the +guest and host sides of the virtio-gpu native-context protocol never drift +apart. Whatever layers Proton and the Steam client on top of it is a closed +build outside this repo, by design — not something this repo names, links +to, or depends on. + +## The nesinit gap + +Nothing in this image starts a payload. The old `nestri-guest-hub` did that +— per its own commit message, the open `neshub` *"loses `--proton`, +`--steamclient-so`, `--root` and the game uid/gid, and no longer ends by +handing the process to a controller... Deciding when the box is finished +belongs to `nesinit`."* `nesinit` — the guest init/session-supervisor that +would actually launch `nescope -- ` and power the box down — is +referenced in commit messages and `nesbox/PROGRESS.md` but does not exist as +open code in either repo. + +So `/etc/init.d/nescope` here starts nescope in **plain-compositor mode** +(no command after `--`): it comes up, provides the Wayland/X11 environment, +and waits for something to connect. That makes the image genuinely bootable +and testable — `neshub`, `neswire`, `nescope` all come up under OpenRC and +you get a real Wayland socket to point a client at — but running an actual +game is still nesinit's job, and nesinit isn't part of this build. Whoever +picks that up next should read `apps/neshub/README.md`'s "What it does not +do" section first. + +## Network defaults + +`/etc/init.d/guest-net` reads `nestri.ip=`/`nestri.gw=` off the kernel +command line, falling back to `172.30.0.2/24` via `172.30.0.1` if neither is +set — nesbox's own default tap addressing. Keep these in step if that +changes on the nesbox side. diff --git a/build/etc/conf.d/agetty.hvc0.debug b/build/etc/conf.d/agetty.hvc0.debug new file mode 100644 index 00000000..79dc110f --- /dev/null +++ b/build/etc/conf.d/agetty.hvc0.debug @@ -0,0 +1,3 @@ +baud="115200" +term_type="vt100" +agetty_options="--autologin root --noclear" diff --git a/build/etc/conf.d/agetty.hvc0.prod b/build/etc/conf.d/agetty.hvc0.prod new file mode 100644 index 00000000..8a9020af --- /dev/null +++ b/build/etc/conf.d/agetty.hvc0.prod @@ -0,0 +1,3 @@ +baud="115200" +term_type="vt100" +agetty_options="--noclear" diff --git a/build/etc/conf.d/nescope b/build/etc/conf.d/nescope new file mode 100644 index 00000000..3ae9f1d2 --- /dev/null +++ b/build/etc/conf.d/nescope @@ -0,0 +1,16 @@ +# Configuration for nescope, the headless compositor. +# +# Sources nestri-user-env rather than symlinking it, for the same reason +# neshub and neswire do. +. /etc/conf.d/nestri-user-env + +RUST_LOG="${RUST_LOG:-nescope=info}" +export RUST_LOG + +# Started in plain-compositor mode by /etc/init.d/nescope (no command after +# `--`): it comes up and waits for something to connect rather than wrapping +# a payload. Starting an actual payload — a game, a desktop — is nesinit's +# job, and nesinit is not open code yet. See build/README.md. +NESCOPE_SOCKET="${NESCOPE_SOCKET:-nescope-0}" +NESCOPE_INPUT_IPC="${NESCOPE_INPUT_IPC:-/tmp/nestri-input.sock}" +export NESCOPE_SOCKET NESCOPE_INPUT_IPC diff --git a/build/etc/conf.d/neshub b/build/etc/conf.d/neshub new file mode 100644 index 00000000..96018062 --- /dev/null +++ b/build/etc/conf.d/neshub @@ -0,0 +1,18 @@ +# Configuration for neshub, the media hub. +# +# Not a symlink to nestri-user-env like pipewire and friends: neshub wants a +# setting of its own, so it sources that file rather than replacing it — same +# pattern as nescope and neswire below. +. /etc/conf.d/nestri-user-env + +# What neshub logs, and nothing else. +# +# Scoped and exported for the reason spelled out at length in the old +# nestri-guest-hub conf.d this replaces: an unscoped `RUST_LOG=info` turns on +# every crate linked in, iroh included, and OpenRC hands a service its own +# environment, not the shell's locals — an unexported variable here goes +# nowhere. +# +# Overridable, so a bad boot can be re-run with more without a rebuild. +RUST_LOG="${RUST_LOG:-neshub=info}" +export RUST_LOG diff --git a/build/etc/conf.d/nestri-user-env b/build/etc/conf.d/nestri-user-env new file mode 100644 index 00000000..13973ba3 --- /dev/null +++ b/build/etc/conf.d/nestri-user-env @@ -0,0 +1,30 @@ +HOME="/home/nestri" +USER="nestri" +LOGNAME="nestri" +XDG_RUNTIME_DIR="/run/user/1000" +XDG_CONFIG_HOME="/home/nestri/.config" +XDG_DATA_HOME="/home/nestri/.local/share" +XDG_CACHE_HOME="/home/nestri/.cache" +XDG_STATE_HOME="/home/nestri/.local/state" +XDG_DATA_DIRS="/usr/local/share:/usr/share" +XDG_SESSION_TYPE="wayland" +XDG_SESSION_DESKTOP="nestri" +DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus" + +# Driver forcing, mirrored from /etc/profile.d/nestri-env.sh. That file only +# runs for login shells, and nothing here is a login shell — every service +# below runs from OpenRC. Capture happens through a Vulkan layer, so a +# process that reached the GPU via native OpenGL would render fine and +# capture nothing; zink is what makes such a process capturable at all. +# These are load-bearing. Keep them in step with the profile.d copy. +#__GLX_VENDOR_LIBRARY_NAME="mesa" +#MESA_LOADER_DRIVER_OVERRIDE="zink" +#GALLIUM_DRIVER="zink" + +nestri_export_env() { + export HOME USER LOGNAME + export XDG_RUNTIME_DIR XDG_CONFIG_HOME XDG_DATA_HOME XDG_CACHE_HOME XDG_STATE_HOME + export XDG_DATA_DIRS XDG_SESSION_TYPE XDG_SESSION_DESKTOP + export DBUS_SESSION_BUS_ADDRESS + #export __GLX_VENDOR_LIBRARY_NAME MESA_LOADER_DRIVER_OVERRIDE GALLIUM_DRIVER +} diff --git a/build/etc/conf.d/neswire b/build/etc/conf.d/neswire new file mode 100644 index 00000000..517d42b2 --- /dev/null +++ b/build/etc/conf.d/neswire @@ -0,0 +1,32 @@ +# Configuration for the audio wire. +# +# Not a symlink to nestri-user-env like pipewire and friends, for the same +# reason neshub is not: it needs a setting of its own -- and it still wants +# that file's environment, so it sources it rather than replacing it. +. /etc/conf.d/nestri-user-env + +# What neswire logs, and nothing else. +# +# Without this neswire is *silent* -- it starts, fails, and exits leaving an +# empty log, which is indistinguishable from never having been started at +# all. That ambiguity is the reason this file exists. +# +# Scoped, and `export`ed, for the reasons spelled out at length in +# conf.d/neshub: an unscoped `info` turns on every linked crate, and an +# unexported one reaches the service's shell and not the service. +# +# Overridable, so a bad boot can be re-run with more without a rebuild. +RUST_LOG="${RUST_LOG:-neswire=info}" +export RUST_LOG + +# neswire's own defaults, spelled out rather than inherited. +# +# All four are clap args with defaults (`neswire --help`), so the service +# works without them. They are here because the IPC path is a contract with +# neshub -- it reads this socket -- and a contract that lives only inside two +# binaries' default values is one nobody can check. +NESWIRE_IPC_PATH="/tmp/nestri-audio.sock" +NESWIRE_CHANNELS="2" +NESWIRE_PACKET_DURATION_MS="5" +NESWIRE_BITRATE_PER_CHANNEL="64" +export NESWIRE_IPC_PATH NESWIRE_CHANNELS NESWIRE_PACKET_DURATION_MS NESWIRE_BITRATE_PER_CHANNEL diff --git a/build/etc/fstab b/build/etc/fstab new file mode 100644 index 00000000..92284f56 --- /dev/null +++ b/build/etc/fstab @@ -0,0 +1,29 @@ +/dev/vda / ext4 rw,relatime 0 1 +devtmpfs /dev devtmpfs rw,nosuid 0 0 +proc /proc proc rw,nosuid,nodev,noexec 0 0 +sysfs /sys sysfs rw,nosuid,nodev,noexec 0 0 +tmpfs /tmp tmpfs rw,nosuid,nodev,size=64M 0 0 +tmpfs /run tmpfs rw,nosuid,nodev,size=32M,mode=0755 0 0 +tmpfs /var/log tmpfs rw,nosuid,nodev,size=16M 0 0 +# POSIX shared memory. devtmpfs does not provide it and nothing else mounts it, +# so without this entry /dev/shm does not exist at all and `shm_open` fails. +# +# PipeWire itself gets by: it allocates buffers with memfd_create and only +# falls back to /dev/shm. Kept as a precaution for anything else that uses +# POSIX shm/semaphores directly — Proton is the known example, and it is not +# part of this image yet (see build/README.md), but a failure here is silent +# rather than fatal, so it costs nothing to have ready. +# +# `nosuid,nodev` and a size cap because everything a game can write to should +# have both. +tmpfs /dev/shm tmpfs rw,nosuid,nodev,size=256M 0 0 +# The guest's logs, on a host directory that outlives the VM. +# +# Here rather than mounted by a service, because the failures worth reading +# are the ones that happen *before* anything else has mounted something — a +# log directory that appears only after a successful start cannot record an +# unsuccessful one. The tag is fixed by nessh, so plain fstab works. +# +# `nofail` because a VM started without the share still has to boot: that is +# how somebody gets a shell to find out why it has no share. +logs /nestri/logs virtiofs rw,nofail 0 0 diff --git a/build/etc/init.d/dbus-session b/build/etc/init.d/dbus-session new file mode 100644 index 00000000..833e5810 --- /dev/null +++ b/build/etc/init.d/dbus-session @@ -0,0 +1,33 @@ +#!/sbin/openrc-run + +description="D-Bus session bus for nestri user" + +nestri_export_env + +command="/usr/bin/dbus-daemon" +command_args="--session --address=unix:path=/run/user/1000/bus --nofork --nopidfile --print-address" +command_user="nestri:nestri" +command_background="yes" +pidfile="/run/nestri/dbus-session.pid" + +depend() { + need xdg-runtime dbus + before pipewire wireplumber +} + +start_pre() { + checkpath -d -m 0755 -o "nestri:nestri" /run/nestri + if [ -e "/run/user/1000/bus" ]; then + ewarn "Removing stale bus socket" + rm -f "/run/user/1000/bus" + fi +} + +start_post() { + local i=0 + while [ ! -S /run/user/1000/bus ] && [ $i -lt 20 ]; do + sleep 0.1 + i=$((i+1)) + done + [ -S /run/user/1000/bus ] || { eerror "Bus didn't appear"; return 1; } +} diff --git a/build/etc/init.d/dbus-system b/build/etc/init.d/dbus-system new file mode 100644 index 00000000..a79a25fb --- /dev/null +++ b/build/etc/init.d/dbus-system @@ -0,0 +1,19 @@ +#!/sbin/openrc-run + +description="D-Bus system message bus" + +command="/usr/bin/dbus-daemon" +command_args="--system --nofork --nopidfile" +command_background="yes" +pidfile="/run/dbus/dbus.pid" + +depend() { + need localmount xdg-runtime + before pipewire +} + +start_pre() { + checkpath -d -m 0755 -o root:root /run/dbus + checkpath -d -m 0755 -o messagebus:messagebus /var/run/dbus 2>/dev/null || true + dbus-uuidgen --ensure=/var/lib/dbus/machine-id +} diff --git a/build/etc/init.d/guest-net b/build/etc/init.d/guest-net new file mode 100644 index 00000000..21aa178a --- /dev/null +++ b/build/etc/init.d/guest-net @@ -0,0 +1,92 @@ +#!/sbin/openrc-run + +description="Configure the guest's network to match the host's tap" + +# Overridden from /etc/conf.d/guest-net if present. These defaults match +# nesbox's own defaults; if you change the `network` section in the VM's +# JSON, change these to match. +: ${GUEST_IFACE:=eth0} +: ${GUEST_IP:=172.30.0.2} +: ${GUEST_PREFIX:=24} +: ${GUEST_GATEWAY:=172.30.0.1} + +depend() { + need localmount + provide net + keyword -shutdown +} + +# Read one `nestri.=` from the kernel command line. +# +# The address has to come from somewhere per-boot, because the alternative -- +# baking it into the image -- makes every guest built from that image the +# same host on the network. Two sandboxes then collide the moment they run +# together. +# +# `nestri.`-prefixed rather than the kernel's own `ip=`: that one needs +# CONFIG_IP_PNP and exists to configure NFS root, and the prefix makes it +# obvious whose parameter this is. +cmdline_value() { + local key="$1" word + for word in $(cat /proc/cmdline 2>/dev/null); do + case "$word" in + "nestri.${key}="*) printf '%s' "${word#nestri.${key}=}"; return 0 ;; + esac + done + return 1 +} + +start() { + ebegin "Bringing up loopback" + ip link set lo up + eend $? + + # The command line wins over conf.d when it says anything, and conf.d is + # the fallback so a hand-written VM config with no parameters keeps + # working -- which is how a guest gets debugged. + local source="/etc/conf.d/guest-net" + local cmdline_ip + if cmdline_ip="$(cmdline_value ip)"; then + # Accepts address/prefix; a bare address keeps the configured prefix + # rather than guessing one. + case "$cmdline_ip" in + */*) + GUEST_IP="${cmdline_ip%%/*}" + GUEST_PREFIX="${cmdline_ip##*/}" + ;; + *) GUEST_IP="$cmdline_ip" ;; + esac + source="kernel command line" + fi + + local cmdline_gw + if cmdline_gw="$(cmdline_value gw)"; then + GUEST_GATEWAY="$cmdline_gw" + source="kernel command line" + fi + + # The VM may have been started with no network device at all, which is a + # perfectly good configuration. Do not fail the boot over it. + if [ ! -e "/sys/class/net/${GUEST_IFACE}" ]; then + einfo "no ${GUEST_IFACE}: this VM has no network device" + return 0 + fi + + # Says which source won, because "the address is wrong" and "the address + # came from somewhere unexpected" look identical from inside the guest. + ebegin "Configuring ${GUEST_IFACE} as ${GUEST_IP}/${GUEST_PREFIX} via ${GUEST_GATEWAY} (from ${source})" + # `replace` rather than `add` so a restart is not an error. + ip link set "${GUEST_IFACE}" up && + ip addr replace "${GUEST_IP}/${GUEST_PREFIX}" dev "${GUEST_IFACE}" && + ip route replace default via "${GUEST_GATEWAY}" dev "${GUEST_IFACE}" + eend $? "could not configure ${GUEST_IFACE}" +} + +stop() { + if [ -e "/sys/class/net/${GUEST_IFACE}" ]; then + ebegin "Bringing down ${GUEST_IFACE}" + ip link set "${GUEST_IFACE}" down + eend 0 + fi + return 0 +} diff --git a/build/etc/init.d/nescope b/build/etc/init.d/nescope new file mode 100644 index 00000000..9c916c00 --- /dev/null +++ b/build/etc/init.d/nescope @@ -0,0 +1,33 @@ +#!/sbin/openrc-run + +description="Nestri headless compositor" + +nestri_export_env + +: "${NESTRI_UID:=1000}" +: "${NESTRI_USER:=nestri}" + +command="/usr/bin/nescope" +# No command after `--`: plain-compositor mode. nescope comes up and waits +# for something to connect rather than wrapping a payload — starting one is +# nesinit's job, and nesinit is not open code yet. See build/README.md. +command_user="${NESTRI_USER}:${NESTRI_USER}" +command_background="yes" +pidfile="/run/nestri/nescope.pid" +output_log="/nestri/logs/nescope.log" +error_log="/nestri/logs/nescope.log" +respawn="yes" +respawn_delay="2" +respawn_max="2" + +export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}" + +depend() { + need xdg-runtime + use neshub + after xdg-runtime +} + +start_pre() { + checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/nestri +} diff --git a/build/etc/init.d/neshub b/build/etc/init.d/neshub new file mode 100644 index 00000000..e408c22b --- /dev/null +++ b/build/etc/init.d/neshub @@ -0,0 +1,39 @@ +#!/sbin/openrc-run + +description="Nestri media hub" + +nestri_export_env + +: "${NESTRI_UID:=1000}" +: "${NESTRI_USER:=nestri}" + +command="/usr/bin/neshub" +# Unprivileged. The old nestri-guest-hub ran as root and mounted the +# session's filesystem itself; the open neshub does neither — per its own +# README it only muxes the Unix sockets the other components dial into one +# QUIC endpoint. Whatever ends up owning session mounts (nesinit, presumably) +# is a separate, still-closed piece. +command_user="${NESTRI_USER}:${NESTRI_USER}" +command_background="yes" +pidfile="/run/nestri/neshub.pid" + +# Where neshub's output goes. /nestri/logs is a virtiofs share mounted from +# /etc/fstab at boot, before this service starts, so a hub that fails +# immediately still leaves a record, and the record survives the VM. +output_log="/nestri/logs/neshub.log" +error_log="/nestri/logs/neshub.log" +respawn="yes" +respawn_delay="2" +respawn_max="2" # NO infinite — if it fails, it fails for a reason + +export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}" + +depend() { + need xdg-runtime + use net + after xdg-runtime +} + +start_pre() { + checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/nestri +} diff --git a/build/etc/init.d/neswire b/build/etc/init.d/neswire new file mode 100644 index 00000000..2c61af9e --- /dev/null +++ b/build/etc/init.d/neswire @@ -0,0 +1,93 @@ +#!/sbin/openrc-run + +description="Nestri pipewire audio sink" + +nestri_export_env + +: "${NESTRI_UID:=1000}" +: "${NESTRI_USER:=nestri}" + +command="/usr/bin/neswire" +command_user="${NESTRI_USER}:${NESTRI_USER}" +command_background="yes" +pidfile="/run/nestri/neswire.pid" +output_log="/nestri/logs/neswire.log" +error_log="/nestri/logs/neswire.log" +respawn="yes" +respawn_delay="2" +respawn_max="2" + +export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}" +export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/${NESTRI_UID}/bus" + +depend() { + # wireplumber is a hard dependency, not a nicety: start_post below pins + # the default sink through the `default` metadata object, and + # WirePlumber is what owns that object. Started concurrently, the pin + # lands in an object pw-metadata created itself, which is destroyed the + # moment it exits. + need xdg-runtime dbus dbus-session pipewire wireplumber + use neshub + after pipewire wireplumber neshub +} + +start_pre() { + checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/nestri +} + +# Point the graph at neswire's sink, by name. +# +# `neswire` registers itself as media.class = Audio/Sink, node.name = neswire +# ("Neswire Cloud Gaming Audio Sink"). With the hardware monitors off it is +# the only sink, so WirePlumber's find-best hook should land on it anyway -- +# this makes it explicit rather than a consequence of there being nothing +# else, so adding a second sink later cannot silently steal the default. +# +# `default.configured.audio.sink` is the metadata WirePlumber's find-selected +# hook reads, and it takes a name; `wpctl set-default` takes a numeric object +# id that changes every boot, which is why this uses pw-metadata instead. +# +# Not fatal if it fails: find-best still has one candidate. A game playing +# into the wrong sink is silent, and a warning here is the only place that +# would say so. +start_post() { + local i=0 + while [ $i -lt 50 ]; do + pw-dump 2>/dev/null | grep -q '"node.name": "neswire"' && break + sleep 0.1 + i=$((i+1)) + done + if ! pw-dump 2>/dev/null | grep -q '"node.name": "neswire"'; then + ewarn "neswire started but its sink never appeared in the graph" + return 0 + fi + + # Wait for WirePlumber to own the `default` metadata before writing to it. + # + # `need wireplumber` only guarantees its script returned, not that it has + # built its objects. Writing too early is silently useless rather than an + # error: pw-metadata creates the object, sets the key, exits 0, and the + # object dies with the client. So wait for the object to exist, and say + # so if it never does. + i=0 + while [ $i -lt 50 ]; do + pw-metadata -n default >/dev/null 2>&1 && break + sleep 0.1 + i=$((i+1)) + done + if ! pw-metadata -n default >/dev/null 2>&1; then + ewarn "no 'default' metadata: WirePlumber is not running, sink not pinned" + return 0 + fi + + pw-metadata -n default 0 default.configured.audio.sink '{ "name": "neswire" }' \ + >/dev/null 2>&1 + + # Read it back. A write that did not stick is the failure this whole + # sequence exists to catch, and it is invisible unless checked. + if pw-metadata -n default 2>/dev/null | grep -q "neswire"; then + einfo "default sink pinned to neswire" + else + ewarn "pinned neswire as default sink but it did not stick" + fi +} diff --git a/build/etc/init.d/pipewire b/build/etc/init.d/pipewire new file mode 100644 index 00000000..e50ded6e --- /dev/null +++ b/build/etc/init.d/pipewire @@ -0,0 +1,45 @@ +#!/sbin/openrc-run + +description="PipeWire multimedia daemon (system mode for nestri)" + +nestri_export_env + +: "${NESTRI_UID:=1000}" +: "${NESTRI_USER:=nestri}" + +command="/usr/bin/pipewire" +command_user="${NESTRI_USER}:${NESTRI_USER}" +command_background="yes" +pidfile="/run/pipewire/pipewire.pid" + +export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}" +export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/${NESTRI_UID}/bus" + +depend() { + need xdg-runtime dbus + use dbus +} + +start_pre() { + checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/pipewire +} + +# `need pipewire` only waits for this script to return, and with +# command_background that is the moment start-stop-daemon forks -- not the +# moment PipeWire is accepting connections. With rc_parallel="YES" every +# client (wireplumber, neswire) then races the socket and a client that +# loses simply exits. +# +# dbus-session already solved exactly this for the bus socket. Same shape +# here. +start_post() { + local i=0 + while [ ! -S "/run/user/${NESTRI_UID}/pipewire-0" ] && [ $i -lt 50 ]; do + sleep 0.1 + i=$((i+1)) + done + [ -S "/run/user/${NESTRI_UID}/pipewire-0" ] || { + eerror "PipeWire socket never appeared" + return 1 + } +} diff --git a/build/etc/init.d/wireplumber b/build/etc/init.d/wireplumber new file mode 100644 index 00000000..1c71732a --- /dev/null +++ b/build/etc/init.d/wireplumber @@ -0,0 +1,20 @@ +#!/sbin/openrc-run + +description="WirePlumber session manager for PipeWire" + +nestri_export_env + +: "${NESTRI_UID:=1000}" +: "${NESTRI_USER:=nestri}" + +command="/usr/bin/wireplumber" +command_user="${NESTRI_USER}:${NESTRI_USER}" +command_background="yes" +pidfile="/run/pipewire/wireplumber.pid" + +export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}" + +depend() { + need pipewire + after pipewire +} diff --git a/build/etc/init.d/xdg-runtime b/build/etc/init.d/xdg-runtime new file mode 100644 index 00000000..8d271047 --- /dev/null +++ b/build/etc/init.d/xdg-runtime @@ -0,0 +1,44 @@ +#!/sbin/openrc-run + +description="Create runtime directories (XDG_RUNTIME_DIR, X11 socket dir..)" + +NESTRI_UID="${NESTRI_UID:-1000}" +NESTRI_USER="${NESTRI_USER:-nestri}" + +depend() { + need localmount + before dbus pipewire +} + +start() { + ebegin "Preparing runtime directories" + + if ! getent passwd "${NESTRI_USER}" >/dev/null 2>&1; then + eerror "User ${NESTRI_USER} does not exist" + return 1 + fi + + # Per-user XDG runtime dir + mkdir -p "/run/user/${NESTRI_UID}" + chown "${NESTRI_USER}:${NESTRI_USER}" "/run/user/${NESTRI_UID}" + chmod 0700 "/run/user/${NESTRI_UID}" + + # X11 socket dir (Xwayland + any X clients expect this) + mkdir -p /tmp/.X11-unix + chown root:root /tmp/.X11-unix + chmod 1777 /tmp/.X11-unix + + # ICE socket dir - some toolkits look for it + mkdir -p /tmp/.ICE-unix + chown root:root /tmp/.ICE-unix + chmod 1777 /tmp/.ICE-unix + + eend $? +} + +stop() { + ebegin "Removing runtime directories" + rm -rf "/run/user/${NESTRI_UID}" + # Don't rm /tmp/.X11-unix on stop - other things may be using it + eend 0 +} diff --git a/build/etc/profile.d/nestri-env.sh b/build/etc/profile.d/nestri-env.sh new file mode 100644 index 00000000..fa0a8590 --- /dev/null +++ b/build/etc/profile.d/nestri-env.sh @@ -0,0 +1,27 @@ +# Nestri user environment — sourced for nestri user logins and su - +if [ "$(id -u)" = "1000" ]; then + export XDG_RUNTIME_DIR="/run/user/1000" + export XDG_CONFIG_HOME="${HOME}/.config" + export XDG_DATA_HOME="${HOME}/.local/share" + export XDG_CACHE_HOME="${HOME}/.cache" + export XDG_STATE_HOME="${HOME}/.local/state" + + export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus" + + # Portal backend selection. "GTK" matches xdg-desktop-portal-gtk. + # Once nescope supports wlr portals, change to "wlroots" or similar. + #export XDG_CURRENT_DESKTOP="${XDG_CURRENT_DESKTOP:-GTK}" + export XDG_SESSION_TYPE="${XDG_SESSION_TYPE:-wayland}" + export XDG_SESSION_DESKTOP="${XDG_SESSION_DESKTOP:-nestri}" + + # Ensure proper VAAPI driver is used + #export LIBVA_DRIVER_NAME="radeonsi" + + # Force zink usage for OpenGL -> Vulkan translation + #export __GLX_VENDOR_LIBRARY_NAME=mesa + #export MESA_LOADER_DRIVER_OVERRIDE=zink + #export GALLIUM_DRIVER=zink + + # Ensure standard XDG dirs exist + mkdir -p "${XDG_CONFIG_HOME}" "${XDG_DATA_HOME}" "${XDG_CACHE_HOME}" "${XDG_STATE_HOME}" 2>/dev/null || true +fi diff --git a/build/etc/rc.conf b/build/etc/rc.conf new file mode 100644 index 00000000..5fc4c575 --- /dev/null +++ b/build/etc/rc.conf @@ -0,0 +1,27 @@ +# Nestri rootfs OpenRC config + +rc_parallel="YES" +rc_depend_strict="NO" +rc_interactive="NO" +rc_shell_timeout="0" +# Every service's start/fail line, on the host directory that outlives the VM. +# +# Without this a guest nobody can log into reports nothing about its own boot: +# a service that was skipped because a dependency failed looks exactly like a +# service that was never registered. /nestri/logs is a virtiofs share, so the +# record survives the guest — see the fstab entry below. +rc_logger="YES" +rc_log_path="/nestri/logs/rc.log" +rc_verbose="NO" + +# cgroup v2 +rc_cgroup_mode="unified" + +# Don't bother trying to set hostname twice +rc_hotplug="!net.*" + +# Faster sulogin behavior on emergency +rc_shell="/bin/sh" + +# Default umask +umask 022 diff --git a/build/etc/resolv.conf b/build/etc/resolv.conf new file mode 100644 index 00000000..bb271869 --- /dev/null +++ b/build/etc/resolv.conf @@ -0,0 +1,2 @@ +nameserver 1.1.1.1 +nameserver 8.8.8.8 diff --git a/build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf b/build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf new file mode 100644 index 00000000..1a679960 --- /dev/null +++ b/build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf @@ -0,0 +1,41 @@ +# WirePlumber, trimmed to what a microVM with no sound card actually needs. +# +# The goal is that `neswire` is the *only* Audio/Sink in the graph, so +# default-node selection has exactly one candidate and cannot pick wrong. +# +# The "Dummy Output" (`auto_null`) sink is not disabled here, and no longer +# needs to be: it came from pipewire-pulse's `module-always-sink`, and +# pipewire-pulse is no longer installed. WirePlumber ships +# scripts/fallback-sink.lua, which creates a node by the same name, but no +# component in wireplumber.conf references it, so it never loads. If auto_null +# ever comes back, it came back with pipewire-pulse -- the switch is +# `pulse.cmd.always-sink = false` in a pipewire-pulse.conf.d drop-in, not +# anything on this side. +# +# pipewire.conf's Dummy-Driver / Freewheel-Driver are a third thing again: +# support.node.driver objects with no ports -- drivers, not sinks, so nothing +# can play into them. Dummy-Driver is load-bearing here. With no ALSA hardware +# it is what clocks the graph, and therefore what paces neswire's sink. Do not +# disable it. + +wireplumber.profiles = { + main = { + # No sound card, no bluetooth adapter, no camera. These only exist to + # probe hardware this guest does not have; disabling them removes the + # only other source of sinks besides neswire. + hardware.audio = disabled + hardware.bluetooth = disabled + hardware.video-capture = disabled + + # The guest root is read-only and every VM is built from the same image, so + # there is nothing to restore and nowhere to save. Left enabled, a stale + # state file is a second, invisible place that can pin a default sink -- + # the exact failure this file exists to rule out. + # + # Equivalent to inheriting mixin.stateless, spelled out so it is greppable. + hooks.default-nodes.state = disabled + hooks.device.profile.state = disabled + hooks.device.routes.state = disabled + hooks.stream.state = disabled + } +} diff --git a/build/scripts/mkimage.sh b/build/scripts/mkimage.sh new file mode 100644 index 00000000..8baf62d5 --- /dev/null +++ b/build/scripts/mkimage.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# Pack a built container image into a raw ext4 disk for nesbox's virtio-blk. +# +# Docker/Buildx has no native "export a raw disk image" step, so this is the +# one part of the pipeline that still has to run outside the Dockerfile. +# +# Run this as yourself, not under sudo. Only mkfs/mount/umount actually need +# root, and those are escalated individually below — `sudo bash mkimage.sh` +# for the whole script is exactly the wrong shape for rootless Podman: the +# image `make build` produced lives in *your* rootless storage, and running +# `podman create` as root afterwards looks in root's separate storage, where +# the tag does not exist. `sudo -v` up front just avoids being prompted +# mid-script for the escalated calls that follow. +set -euo pipefail + +IMAGE="${1:?usage: mkimage.sh [size]}" +OUT="${2:?usage: mkimage.sh [size]}" +SIZE="${3:-5G}" + +if [[ "$(id -u)" -eq 0 ]]; then + echo "mkimage.sh should run as yourself, not root/sudo — see the comment at the top of this script" >&2 + exit 1 +fi + +CONTAINER_RT="$(command -v docker || command -v podman || true)" +[[ -n "$CONTAINER_RT" ]] || { echo "Neither docker nor podman found in PATH" >&2; exit 1; } + +sudo -v # cache credentials once, rather than prompting mid-pipeline + +WORK="$(mktemp -d)" +cleanup() { + mountpoint -q "$WORK/mnt" 2>/dev/null && sudo umount "$WORK/mnt" + rm -rf "$WORK" +} +trap cleanup EXIT + +echo "Exporting ${IMAGE}..." +cid="$("$CONTAINER_RT" create "$IMAGE")" +"$CONTAINER_RT" export "$cid" -o "$WORK/rootfs.tar" +"$CONTAINER_RT" rm -f "$cid" >/dev/null + +echo "Creating ${SIZE} ext4 image at ${OUT}..." +mkdir -p "$(dirname "$OUT")" +truncate -s "$SIZE" "$OUT" +sudo mkfs.ext4 -q -L nestri-root "$OUT" + +mkdir -p "$WORK/mnt" +sudo mount -o loop "$OUT" "$WORK/mnt" +# Same excludes as the old bootstrap extraction: .dockerenv is Docker's own +# marker file, and /dev is devtmpfs at boot, populated by the kernel — a +# tarred copy of the build container's /dev would just be dead weight. +# +# Root, deliberately: the rootfs's own files are owned by root (that is +# correct — it is the guest's root filesystem), and only root can write +# root-owned files onto the loop-mounted ext4. +sudo tar -xf "$WORK/rootfs.tar" -C "$WORK/mnt" --exclude='.dockerenv' --exclude='dev/*' +sudo umount "$WORK/mnt" + +# The image *file* itself was created by `truncate` above as the invoking +# user and never needs to change hands — mkfs/mount/umount touch its +# contents, not its ownership. Asserted, not assumed, since a stray `sudo` +# reordering above would silently hand root ownership of a file the rest of +# this pipeline expects to read and delete without sudo. +[[ "$(stat -c '%U' "$OUT")" == "$(id -un)" ]] || { + echo "warning: ${OUT} is not owned by $(id -un) — sudo chown $(id -un) ${OUT}" >&2 +} +echo "Wrote ${OUT}"