mirror of
https://github.com/nestriness/nestri.git
synced 2026-09-19 17:25:19 +03:00
feat(build): borealis-style multi-stage rootfs for the open guest components (#309)
## What
Adds `build/` — a Dockerfile with `mesa-build`, `nestri-build`,
`os-base`, `runtime`, `runtime_prod` and `runtime_debug` stages, plus
the `etc/` overlay, `mkimage.sh` and a `Makefile` — laid out the way
[borealis](https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/main/project-borealis)
lays out its own `build/`. This moves the guest rootfs formula into this
repo, targeting the four open guest components already here: `nescope`,
`neshub`, `neswire`, `nescapture`.
## Two structural properties worth calling out
- **No privileged host chroot.** A bare `chroot` into a hand-extracted
rootfs needs `/proc`, `/sys`, `/dev` bind-mounted in first. `os-base`
here is `FROM artixlinux/artixlinux:base-openrc` directly with `pacman
-S` as plain `RUN` steps — a Docker build step already has its own
`/proc`/`/sys`/`/dev`.
- **No host-side ownership bug to guard against.** `COPY --from=` runs
as root inside the build with no invoking-user uid in the loop.
## Scope boundary
**Deliberately excludes Proton and Valve's `steamclient.so`** — both
closed, and `CLAUDE.md` forbids closed content in this repo.
`runtime_prod`, tagged `nestrilabs/nestri:base`, is a complete,
bootable, Steam-less image — and also the shared foundation other builds
start from. Whatever layers Proton/Steam on top of it is a closed build
outside this repo, by design.
## Known gap
Nothing starts a payload yet — `nesinit` isn't open code — so
`/etc/init.d/nescope` boots it in plain-compositor mode (no command
after `--`) rather than running a game. Real and testable, just not a
full session yet. Details in `build/README.md`.
## Status
Built and tagged locally as `nestrilabs/nestri:base` (podman, no
`--no-cache` issues, greptile's three findings all fixed and verified
against a real build). Not yet packed into a disk image or run inside
nesbox.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
The PR adds a multi-stage Artix/OpenRC guest-rootfs build for the open
Nestri components, with production and debug image flavors.
- Builds patched Mesa and the Rust workspace in dedicated builder
stages.
- Assembles and configures the bootable guest environment and OpenRC
services.
- Packs a selected container image into an ext4 root filesystem while
retaining rootless container storage access.
<h3>Confidence Score: 5/5</h3>
The PR appears safe to merge.
No blocking failure remains.
<h3>Important Files Changed</h3>
| Filename | Overview |
|----------|----------|
| build/Dockerfile | Defines the complete multi-stage build, overlays
repository-root-relative configuration paths, and creates production and
debug runtime targets. |
| build/Makefile | Provides consistent image build and packing targets
using a repository-root context and matching image tags. |
| build/scripts/mkimage.sh | Keeps container-runtime operations in the
invoking user's storage while escalating only filesystem creation and
mounting operations. |
| build/etc/conf.d/nestri-user-env | Supplies the shared service
environment and export function required by the OpenRC service scripts.
|
| build/etc/init.d/guest-net | Configures optional guest networking from
kernel parameters or stable defaults. |
| build/etc/init.d/neswire | Starts the audio sink after its
dependencies and pins it as the PipeWire default once the graph is
ready. |
<h3>Flowchart</h3>
```mermaid
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Arch builder] --> B[Mesa build]
A --> C[Nestri workspace build]
D[Artix OpenRC base] --> E[Common runtime]
B --> E
C --> E
F[build/etc overlay] --> E
E --> G[runtime_prod]
E --> H[runtime_debug]
G --> I[Container image]
H --> J[Debug container image]
I --> K[mkimage.sh]
J --> K
K --> L[ext4 rootfs]
```
<sub>Reviews (6): Last reviewed commit: ["refactor(build): rename the
published
im..."](6fecc8cc31)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=58981739)</sub>
<!-- /greptile_comment -->
This commit is contained in:
295
build/Dockerfile
Normal file
295
build/Dockerfile
Normal file
@@ -0,0 +1,295 @@
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# nestri guest rootfs — the open half
|
||||
#
|
||||
# Builds a bootable Artix/OpenRC image containing Mesa (virtio-gpu native
|
||||
# context) and the four open guest components: nescope, neshub, neswire,
|
||||
# nescapture. Two leaf targets, selected with `--target`:
|
||||
#
|
||||
# runtime_prod stripped, root locked (default: `make build`)
|
||||
# runtime_debug debug tools, autologin root (`make build-debug`)
|
||||
#
|
||||
# What is deliberately NOT here: Proton, Valve's steamclient.so, or anything
|
||||
# else closed. nestri/CLAUDE.md is explicit that nothing closed enters this
|
||||
# repo. Whatever layers those on top of runtime_prod is a closed build
|
||||
# outside this repo — see build/README.md.
|
||||
#
|
||||
# Build from the repo root, not from build/:
|
||||
# docker build -f build/Dockerfile --target runtime_prod -t nestri-guest .
|
||||
# (`make build` in this directory does exactly that.)
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# initial / builder — Arch, not Artix
|
||||
#
|
||||
# The guest is Artix, but Artix's repos are Arch-derived and the toolchain/
|
||||
# glibc generation is the same, while the archlinux image is the
|
||||
# better-maintained of the two for actually compiling things. Only build
|
||||
# artifacts leave these stages.
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM docker.io/archlinux:base-devel AS initial
|
||||
RUN pacman -Syu --noconfirm
|
||||
|
||||
FROM initial AS builder
|
||||
RUN pacman -S --noconfirm --needed \
|
||||
cmake meson ninja git pkgconf \
|
||||
python python-mako python-yaml python-packaging python-ply \
|
||||
bison flex \
|
||||
libpciaccess libepoxy libglvnd \
|
||||
libx11 libxext libxrandr libxshmfence libxfixes libxxf86vm libxcb \
|
||||
xcb-util-keysyms xorgproto \
|
||||
wayland wayland-protocols \
|
||||
expat zlib zstd libxml2 lm_sensors \
|
||||
llvm clang libclc spirv-tools spirv-llvm-translator glslang \
|
||||
elfutils libva libdrm directx-headers \
|
||||
rust rust-bindgen cbindgen \
|
||||
curl openssl \
|
||||
pixman libxkbcommon \
|
||||
vulkan-headers vulkan-icd-loader \
|
||||
pipewire shaderc opus \
|
||||
libinput \
|
||||
&& pacman -Scc --noconfirm
|
||||
WORKDIR /build
|
||||
ENV ARTIFACTS=/artifacts
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# Mesa — the only piece still fetched from outside this tree
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM builder AS mesa-build
|
||||
|
||||
ARG MESA_GIT=https://gitlab.freedesktop.org/mesa/mesa.git
|
||||
ARG MESA_COMMIT=b78fc73dd898a7dfa87448a4b5ff4459a870e21a
|
||||
|
||||
RUN git clone --depth=1 --revision="${MESA_COMMIT}" "${MESA_GIT}" /build/mesa-src && \
|
||||
cd /build/mesa-src && \
|
||||
meson setup builddir \
|
||||
-Dprefix=/usr \
|
||||
-Dbuildtype=release \
|
||||
-Dplatforms=wayland,x11 \
|
||||
-Dgallium-drivers=zink,radeonsi,iris \
|
||||
-Dvulkan-drivers=amd,intel \
|
||||
-Damdgpu-virtio=true \
|
||||
-Dintel-virtio-experimental=true \
|
||||
-Dvideo-codecs=all \
|
||||
-Degl=enabled \
|
||||
-Dglx=dri \
|
||||
-Dgles1=enabled \
|
||||
-Dgles2=enabled \
|
||||
-Dgbm=enabled \
|
||||
-Dgallium-va=enabled \
|
||||
-Db_ndebug=true && \
|
||||
ninja -C builddir && \
|
||||
DESTDIR=/artifacts/mesa ninja -C builddir install && \
|
||||
rm -rf /build/mesa-src && \
|
||||
find /artifacts/mesa -type f -printf '/%P\n' > /artifacts/mesa/.manifest
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# nestri workspace — same repo now, so this is COPY, not a private clone
|
||||
#
|
||||
# One `cargo build --release --workspace` rather than one stage per binary:
|
||||
# that per-repo splitting existed because nescope/neswire/nescapture/the hub
|
||||
# were four separate private repos and a stage boundary was the only way to
|
||||
# stop bumping one from invalidating the others' build cache. They are one
|
||||
# Cargo workspace with one Cargo.lock now, so a BuildKit cache mount on
|
||||
# target/ gives the same isolation — cargo's own incremental compiler
|
||||
# already knows nescope changing does not touch nesprotocol's .rlib — without
|
||||
# four copies of every shared dependency getting compiled once per stage.
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM builder AS nestri-src
|
||||
WORKDIR /build/nestri
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY crates/nesprotocol crates/nesprotocol
|
||||
COPY apps/nescope apps/nescope
|
||||
COPY apps/neshub apps/neshub
|
||||
COPY apps/neswire apps/neswire
|
||||
COPY apps/nescapture apps/nescapture
|
||||
|
||||
FROM nestri-src AS nestri-build
|
||||
RUN --mount=type=cache,target=/root/.cargo/registry \
|
||||
--mount=type=cache,target=/build/nestri/target \
|
||||
cargo build --release --workspace && \
|
||||
mkdir -p /artifacts/nestri/usr/bin /artifacts/nestri/usr/lib \
|
||||
/artifacts/nestri/usr/share/vulkan/implicit_layer.d && \
|
||||
install -Dm755 target/release/nescope /artifacts/nestri/usr/bin/nescope && \
|
||||
install -Dm755 target/release/neshub /artifacts/nestri/usr/bin/neshub && \
|
||||
install -Dm755 target/release/neswire /artifacts/nestri/usr/bin/neswire && \
|
||||
install -Dm755 target/release/libnescapture_layer.so \
|
||||
/artifacts/nestri/usr/lib/libnescapture_layer.so && \
|
||||
install -Dm644 apps/nescapture/manifest/VK_LAYER_nescapture.json \
|
||||
/artifacts/nestri/usr/share/vulkan/implicit_layer.d/VK_LAYER_nescapture.json && \
|
||||
find /artifacts/nestri -type f -printf '/%P\n' > /artifacts/nestri/.manifest
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# os-base — the Artix rootfs itself
|
||||
#
|
||||
# `FROM artixlinux/artixlinux:base-openrc` directly, and `pacman -S` as plain
|
||||
# RUN steps — not a privileged host `chroot` into a hand-extracted tarball,
|
||||
# which would need /proc, /sys and /dev bind-mounted in first (they don't
|
||||
# exist inside a chroot target until something puts them there). A
|
||||
# Dockerfile RUN step already executes inside a real container with its own
|
||||
# /proc, /sys, /dev, so there is no bind-mount step to write at all.
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM docker.io/artixlinux/artixlinux:base-openrc AS os-base
|
||||
|
||||
RUN pacman -Syu --noconfirm --needed \
|
||||
base openrc udev dbus dbus-openrc \
|
||||
iptables iproute2 \
|
||||
mesa libglvnd libdrm libepoxy libxxf86vm libinput wayland \
|
||||
expat zlib llvm-libs elfutils libva shaderc vulkan-icd-loader \
|
||||
pixman libxkbcommon xcb-util-keysyms xorg-xwayland \
|
||||
pipewire pipewire-audio wireplumber dbus logrotate opus \
|
||||
&& pacman -Scc --noconfirm
|
||||
|
||||
# groupadd -f so this is idempotent whether or not udev's rules already
|
||||
# created these.
|
||||
RUN groupadd -f audio && groupadd -f video && groupadd -f input && groupadd -f render && \
|
||||
useradd -m -u 1000 -s /bin/bash nestri && \
|
||||
for g in audio video input render; do gpasswd -a nestri "$g" >/dev/null; done
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# runtime — everything common to debug and prod
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM os-base AS runtime
|
||||
|
||||
# This is what GHCR actually uses to connect a pushed image back to its
|
||||
# repo — not a setting to toggle after the fact, a label the image has to
|
||||
# carry. Without it a manually-pushed image shows no "used by" repo on its
|
||||
# package page even though this Dockerfile is exactly what built it.
|
||||
LABEL org.opencontainers.image.source="https://github.com/nestrilabs/nestri"
|
||||
|
||||
# Our own builds, overlaid on the distro's mesa. The distro package landed
|
||||
# first (above) so every runtime dependency of *a* Mesa is present and
|
||||
# correctly versioned; this overwrites its .so files with ours.
|
||||
#
|
||||
# COPY --from runs as root inside this build with no invoking-user uid to
|
||||
# stamp onto / or /usr/bin, unlike a host-side `podman cp` + `cp -a` — so
|
||||
# there is no ownership-sanity-check to write here. Nothing to catch, on
|
||||
# purpose, not an oversight.
|
||||
COPY --from=mesa-build /artifacts/mesa/.manifest /tmp/mesa.manifest
|
||||
COPY --from=nestri-build /artifacts/nestri/.manifest /tmp/nestri.manifest
|
||||
RUN cat /tmp/mesa.manifest /tmp/nestri.manifest > /tmp/.strip-manifest && \
|
||||
rm -f /tmp/mesa.manifest /tmp/nestri.manifest
|
||||
COPY --from=mesa-build /artifacts/mesa /
|
||||
COPY --from=nestri-build /artifacts/nestri /
|
||||
RUN ldconfig
|
||||
|
||||
COPY build/etc/ /etc/
|
||||
RUN chmod +x /etc/init.d/*
|
||||
|
||||
# dbus-session, pipewire and wireplumber carry no conf.d of their own — they
|
||||
# just want the shared environment, so their conf.d is a symlink to it rather
|
||||
# than a copy. nescope, neshub and neswire are deliberately absent from this
|
||||
# loop: each has a conf.d file of its own (already landed by the COPY above)
|
||||
# that sources nestri-user-env, because each needs settings the shared file
|
||||
# does not carry. Symlinking them here would overwrite those.
|
||||
RUN for svc in dbus-session pipewire wireplumber; do \
|
||||
ln -sf nestri-user-env "/etc/conf.d/${svc}"; \
|
||||
done
|
||||
|
||||
RUN echo nesbox > /etc/hostname && dbus-uuidgen --ensure=/etc/machine-id
|
||||
|
||||
# Session mount points. The guest root is read-only at runtime, so a runtime
|
||||
# mkdir gets EROFS and takes a service down before it starts — these have to
|
||||
# already exist in the image.
|
||||
RUN mkdir -p /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \
|
||||
chmod 0755 /nestri /nestri/install /nestri/user /nestri/work /nestri/game /nestri/logs && \
|
||||
mkdir -p /dev/shm && chmod 1777 /dev/shm && \
|
||||
rm -f /etc/network/interfaces
|
||||
|
||||
# Serial console: the only way into a guest that will not boot.
|
||||
RUN rm -f /etc/inittab && \
|
||||
ln -sf agetty /etc/init.d/agetty.hvc0 && \
|
||||
{ grep -qx hvc0 /etc/securetty || echo hvc0 >> /etc/securetty; }
|
||||
|
||||
# Every init.d script calling nestri_export_env needs a conf.d that actually
|
||||
# defines it. Worth a build-time check: the failure at runtime is nearly
|
||||
# invisible — OpenRC sources the script, the undefined function is a
|
||||
# "command not found" on stderr, sourcing still returns 0, and the service
|
||||
# starts anyway with HOME and every XDG_* unset.
|
||||
RUN missing=""; \
|
||||
for svc_script in /etc/init.d/*; do \
|
||||
grep -qE '^[[:space:]]*nestri_export_env\b' "$svc_script" 2>/dev/null || continue; \
|
||||
svc="$(basename "$svc_script")"; \
|
||||
. "/etc/conf.d/$svc" >/dev/null 2>&1; \
|
||||
command -v nestri_export_env >/dev/null 2>&1 || missing="$missing $svc"; \
|
||||
done; \
|
||||
[ -z "$missing" ] || { echo "init scripts call nestri_export_env with no conf.d providing it:$missing" >&2; exit 1; }
|
||||
|
||||
# ── OpenRC service registration ──
|
||||
# sysinit
|
||||
RUN rc-update add devfs sysinit && \
|
||||
rc-update add dmesg sysinit && \
|
||||
rc-update add udev sysinit && \
|
||||
rc-update add udev-trigger sysinit && \
|
||||
(rc-update del kmod-static-nodes sysinit || true)
|
||||
|
||||
# boot — guest-net replaces the distro's networking scripts entirely; it
|
||||
# declares `provide net` and brings up lo itself.
|
||||
RUN (rc-update del networking boot || true) && \
|
||||
(rc-update del network-async boot || true) && rm -f /etc/init.d/network-async && \
|
||||
rc-update add guest-net boot && \
|
||||
(rc-update del net.lo boot || true) && \
|
||||
(rc-update del netmount boot || true) && \
|
||||
(rc-update del netmount default || true) && \
|
||||
rc-update add hostname boot && \
|
||||
rc-update add xdg-runtime boot && \
|
||||
rc-update add cgroups boot && \
|
||||
(rc-update del syslog boot || true)
|
||||
|
||||
# Boot-runlevel services with nothing to do in a freshly-built microVM: no
|
||||
# physical console, no swap, and fsck would be checking a filesystem nobody
|
||||
# has touched.
|
||||
RUN for svc in fsck keymaps save-keymaps termencoding save-termencoding swap binfmt seedrng hwclock swclock; do \
|
||||
(rc-update del "$svc" boot || true); \
|
||||
(rc-update del "$svc" shutdown || true); \
|
||||
done
|
||||
|
||||
# default — the user-facing stack. nescope is registered in plain-compositor
|
||||
# mode (no payload after `--`): it comes up and waits for something to
|
||||
# connect. Actually starting a payload is nesinit's job, and nesinit is not
|
||||
# open code yet — see build/README.md.
|
||||
RUN rc-update add agetty.hvc0 default && \
|
||||
for n in 1 2 3 4 5 6; do (rc-update del "agetty.tty${n}" default || true); done && \
|
||||
rc-update add dbus default && \
|
||||
rc-update add dbus-session default && \
|
||||
rc-update add pipewire default && \
|
||||
rc-update add wireplumber default && \
|
||||
rc-update add neshub default && \
|
||||
rc-update add neswire default && \
|
||||
rc-update add nescope default && \
|
||||
rm -f /etc/init.d/shared-root && (rc-update del shared-root boot || true)
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# runtime_prod — the default: `make build`
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM runtime AS runtime_prod
|
||||
|
||||
RUN passwd -l root
|
||||
COPY build/etc/conf.d/agetty.hvc0.prod /etc/conf.d/agetty.hvc0
|
||||
|
||||
RUN while IFS= read -r f; do \
|
||||
[ -f "$f" ] && strip --strip-unneeded "$f" 2>/dev/null || true; \
|
||||
done < /tmp/.strip-manifest; \
|
||||
rm -rf /tmp/.strip-manifest /var/cache/pacman/pkg/* /tmp/* /root/.cache \
|
||||
/usr/share/man /usr/share/doc /usr/share/locale \
|
||||
/usr/lib/cmake /usr/lib/pkgconfig /usr/share/pkgconfig /usr/include
|
||||
|
||||
RUN echo "NESTRI_STAGE=runtime_prod" >> /etc/os-release
|
||||
|
||||
|
||||
# ───────────────────────────────────────────────────────────
|
||||
# runtime_debug — `make build-debug`
|
||||
# ───────────────────────────────────────────────────────────
|
||||
FROM runtime AS runtime_debug
|
||||
|
||||
RUN pacman -Syu --noconfirm --needed vulkan-tools mesa-utils libva-utils && \
|
||||
pacman -Scc --noconfirm
|
||||
RUN echo 'root:nestri' | chpasswd
|
||||
COPY build/etc/conf.d/agetty.hvc0.debug /etc/conf.d/agetty.hvc0
|
||||
|
||||
RUN echo "NESTRI_STAGE=runtime_debug" >> /etc/os-release
|
||||
66
build/Makefile
Normal file
66
build/Makefile
Normal file
@@ -0,0 +1,66 @@
|
||||
SHELL := /bin/bash
|
||||
.PHONY: build build-debug image image-debug clean help
|
||||
|
||||
CONTAINER_RT := $(shell command -v docker 2>/dev/null || command -v podman 2>/dev/null)
|
||||
ifeq ($(CONTAINER_RT),)
|
||||
$(error "Neither docker nor podman found in PATH")
|
||||
endif
|
||||
|
||||
# The Dockerfile COPYs from apps/ and crates/, so the build context is the
|
||||
# repo root, not this directory — same reason borealis's build/ *is* its own
|
||||
# context: here the guest source lives one level up instead of inside build/.
|
||||
CONTEXT := ..
|
||||
# `ghcr.io/nestrilabs/nestri/base` (a package name with a `/` in it, matching
|
||||
# how every other image this org has published is named) is a deliberate
|
||||
# name, not just a tag: this is the image other builds start FROM —
|
||||
# nesbox's jailer image extracts Mesa/virgl from it to keep the guest and
|
||||
# host sides of the virtio-gpu native-context protocol on the same patched
|
||||
# Mesa, and closed downstream builds layer Proton/Steam on top of it
|
||||
# elsewhere. `runtime_prod`/`runtime_debug` already are exactly that: distro
|
||||
# packages plus our own Mesa/nestri artifacts overlaid on top, nothing
|
||||
# stripped that a downstream COPY --from= would miss — no separate
|
||||
# unstripped tag is needed for this.
|
||||
#
|
||||
# The registry host is part of the name on purpose, including for local
|
||||
# builds: a bare `nestrilabs/nestri` has no host, so anyone who pulls
|
||||
# instead of building locally resolves it against Docker Hub by default,
|
||||
# where it does not exist. One name, always pullable, beats a local-only
|
||||
# short name and a different published one.
|
||||
IMAGE_NAME := ghcr.io/nestrilabs/nestri/base
|
||||
OUTPUT_DIR := output
|
||||
ROOTFS_SIZE ?= 5G
|
||||
FORCE_REBUILD ?=
|
||||
|
||||
build:
|
||||
DOCKER_BUILDKIT=1 $(CONTAINER_RT) build $(if $(FORCE_REBUILD),--no-cache,) \
|
||||
-f Dockerfile -t $(IMAGE_NAME):latest --target runtime_prod $(CONTEXT)
|
||||
|
||||
build-debug:
|
||||
DOCKER_BUILDKIT=1 $(CONTAINER_RT) build $(if $(FORCE_REBUILD),--no-cache,) \
|
||||
-f Dockerfile -t $(IMAGE_NAME):debug --target runtime_debug $(CONTEXT)
|
||||
|
||||
# Not `sudo make image`/sudo'd in here: mkimage.sh runs as you and escalates
|
||||
# only the specific commands that need root. Under rootless Podman, `make
|
||||
# build` stores the image in *your* storage — sudo-ing the whole script would
|
||||
# have root's podman look for that tag in its own, separate storage and fail
|
||||
# to find it.
|
||||
image: build
|
||||
@mkdir -p $(OUTPUT_DIR)
|
||||
bash scripts/mkimage.sh $(IMAGE_NAME):latest $(OUTPUT_DIR)/rootfs.ext4 $(ROOTFS_SIZE)
|
||||
|
||||
image-debug: build-debug
|
||||
@mkdir -p $(OUTPUT_DIR)
|
||||
bash scripts/mkimage.sh $(IMAGE_NAME):debug $(OUTPUT_DIR)/rootfs-debug.ext4 $(ROOTFS_SIZE)
|
||||
|
||||
clean:
|
||||
rm -rf $(OUTPUT_DIR)
|
||||
|
||||
help:
|
||||
@echo "Usage:"
|
||||
@echo " make build Build the runtime_prod container image"
|
||||
@echo " make build-debug Build the runtime_debug container image"
|
||||
@echo " make image Build + pack runtime_prod into output/rootfs.ext4"
|
||||
@echo " make image-debug Build + pack runtime_debug into output/rootfs-debug.ext4"
|
||||
@echo " make clean Remove output/"
|
||||
@echo " make FORCE_REBUILD=1 ... Rebuild from scratch, no layer cache"
|
||||
@echo " make ROOTFS_SIZE=8G image Override the packed image size (default 5G)"
|
||||
90
build/README.md
Normal file
90
build/README.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# build/ — the guest rootfs
|
||||
|
||||
Builds a bootable Artix/OpenRC image for the box's virtio-blk root: Mesa
|
||||
(virtio-gpu native context) plus the four open guest components —
|
||||
[`nescope`](../apps/nescope), [`neshub`](../apps/neshub),
|
||||
[`neswire`](../apps/neswire), [`nescapture`](../apps/nescapture) — laid out
|
||||
the way [borealis](https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/main/project-borealis)
|
||||
lays out its `build/`: one big multi-stage `Dockerfile`, `--target` picks the
|
||||
flavor, `etc/` holds the files that get overlaid onto the image verbatim.
|
||||
|
||||
```
|
||||
build/
|
||||
├── Dockerfile everything, in stages: mesa-build, nestri-build,
|
||||
│ os-base, runtime, runtime_prod, runtime_debug
|
||||
├── etc/ overlaid onto the image's /etc as-is
|
||||
├── scripts/
|
||||
│ └── mkimage.sh docker export → raw ext4, for nesbox's virtio-blk
|
||||
├── Makefile
|
||||
└── output/ `make image` writes here (gitignored)
|
||||
```
|
||||
|
||||
```sh
|
||||
make build # docker build --target runtime_prod → ghcr.io/nestrilabs/nestri/base:latest
|
||||
make build-debug # docker build --target runtime_debug → ghcr.io/nestrilabs/nestri/base:debug
|
||||
make image # + pack into output/rootfs.ext4
|
||||
make image-debug # + pack into output/rootfs-debug.ext4
|
||||
```
|
||||
|
||||
## Design notes
|
||||
|
||||
Three things worth knowing about how this is put together:
|
||||
|
||||
1. **No privileged host chroot.** A bare `chroot` into a hand-extracted
|
||||
rootfs needs `/proc`, `/sys`, `/dev` bind-mounted in first — they don't
|
||||
exist inside a chroot target until something puts them there. `os-base`
|
||||
here is `FROM artixlinux/artixlinux:base-openrc` directly, with
|
||||
`pacman -S` as plain `RUN` steps — a Docker build step already runs
|
||||
inside a real container with its own `/proc`, `/sys`, `/dev`, so that
|
||||
whole bind-mount mechanism has nothing to do.
|
||||
|
||||
2. **No host-side ownership bug to guard against.** `COPY --from=` runs as
|
||||
root inside the build with no host user in the loop, so there's no
|
||||
invoking-user uid getting stamped onto `/`, `/usr/bin`, or anywhere else
|
||||
a build step touches — a failure mode some overlay approaches need an
|
||||
explicit sanity check for doesn't exist here to check for.
|
||||
|
||||
3. **One `cargo build --release --workspace`, not one stage per binary.**
|
||||
`nescope`, `neshub`, `neswire` and `nescapture` share one Cargo workspace
|
||||
and one `Cargo.lock` — a BuildKit cache mount on `target/` gives cargo's
|
||||
own incremental compiler per-crate isolation without needing a separate
|
||||
Docker stage (and a separate full rebuild of `nesprotocol`) per binary.
|
||||
|
||||
**What is deliberately not here: Proton, and Valve's `steamclient.so`.**
|
||||
`nestri/CLAUDE.md` is explicit — *"Nothing closed may enter this repo. Not
|
||||
source, not a dependency, not a directory that 'looked convenient'."* Both
|
||||
are closed. `runtime_prod` from this Dockerfile — tagged
|
||||
`ghcr.io/nestrilabs/nestri/base:latest` — is a complete, bootable, Steam-less guest image,
|
||||
and also the shared foundation other builds start from: nesbox's jailer
|
||||
image (see `nesbox/build/`) extracts Mesa and virglrenderer from it so the
|
||||
guest and host sides of the virtio-gpu native-context protocol never drift
|
||||
apart. Whatever layers Proton and the Steam client on top of it is a closed
|
||||
build outside this repo, by design — not something this repo names, links
|
||||
to, or depends on.
|
||||
|
||||
## The nesinit gap
|
||||
|
||||
Nothing in this image starts a payload. The old `nestri-guest-hub` did that
|
||||
— per its own commit message, the open `neshub` *"loses `--proton`,
|
||||
`--steamclient-so`, `--root` and the game uid/gid, and no longer ends by
|
||||
handing the process to a controller... Deciding when the box is finished
|
||||
belongs to `nesinit`."* `nesinit` — the guest init/session-supervisor that
|
||||
would actually launch `nescope -- <payload>` and power the box down — is
|
||||
referenced in commit messages and `nesbox/PROGRESS.md` but does not exist as
|
||||
open code in either repo.
|
||||
|
||||
So `/etc/init.d/nescope` here starts nescope in **plain-compositor mode**
|
||||
(no command after `--`): it comes up, provides the Wayland/X11 environment,
|
||||
and waits for something to connect. That makes the image genuinely bootable
|
||||
and testable — `neshub`, `neswire`, `nescope` all come up under OpenRC and
|
||||
you get a real Wayland socket to point a client at — but running an actual
|
||||
game is still nesinit's job, and nesinit isn't part of this build. Whoever
|
||||
picks that up next should read `apps/neshub/README.md`'s "What it does not
|
||||
do" section first.
|
||||
|
||||
## Network defaults
|
||||
|
||||
`/etc/init.d/guest-net` reads `nestri.ip=`/`nestri.gw=` off the kernel
|
||||
command line, falling back to `172.30.0.2/24` via `172.30.0.1` if neither is
|
||||
set — nesbox's own default tap addressing. Keep these in step if that
|
||||
changes on the nesbox side.
|
||||
3
build/etc/conf.d/agetty.hvc0.debug
Normal file
3
build/etc/conf.d/agetty.hvc0.debug
Normal file
@@ -0,0 +1,3 @@
|
||||
baud="115200"
|
||||
term_type="vt100"
|
||||
agetty_options="--autologin root --noclear"
|
||||
3
build/etc/conf.d/agetty.hvc0.prod
Normal file
3
build/etc/conf.d/agetty.hvc0.prod
Normal file
@@ -0,0 +1,3 @@
|
||||
baud="115200"
|
||||
term_type="vt100"
|
||||
agetty_options="--noclear"
|
||||
16
build/etc/conf.d/nescope
Normal file
16
build/etc/conf.d/nescope
Normal file
@@ -0,0 +1,16 @@
|
||||
# Configuration for nescope, the headless compositor.
|
||||
#
|
||||
# Sources nestri-user-env rather than symlinking it, for the same reason
|
||||
# neshub and neswire do.
|
||||
. /etc/conf.d/nestri-user-env
|
||||
|
||||
RUST_LOG="${RUST_LOG:-nescope=info}"
|
||||
export RUST_LOG
|
||||
|
||||
# Started in plain-compositor mode by /etc/init.d/nescope (no command after
|
||||
# `--`): it comes up and waits for something to connect rather than wrapping
|
||||
# a payload. Starting an actual payload — a game, a desktop — is nesinit's
|
||||
# job, and nesinit is not open code yet. See build/README.md.
|
||||
NESCOPE_SOCKET="${NESCOPE_SOCKET:-nescope-0}"
|
||||
NESCOPE_INPUT_IPC="${NESCOPE_INPUT_IPC:-/tmp/nestri-input.sock}"
|
||||
export NESCOPE_SOCKET NESCOPE_INPUT_IPC
|
||||
18
build/etc/conf.d/neshub
Normal file
18
build/etc/conf.d/neshub
Normal file
@@ -0,0 +1,18 @@
|
||||
# Configuration for neshub, the media hub.
|
||||
#
|
||||
# Not a symlink to nestri-user-env like pipewire and friends: neshub wants a
|
||||
# setting of its own, so it sources that file rather than replacing it — same
|
||||
# pattern as nescope and neswire below.
|
||||
. /etc/conf.d/nestri-user-env
|
||||
|
||||
# What neshub logs, and nothing else.
|
||||
#
|
||||
# Scoped and exported for the reason spelled out at length in the old
|
||||
# nestri-guest-hub conf.d this replaces: an unscoped `RUST_LOG=info` turns on
|
||||
# every crate linked in, iroh included, and OpenRC hands a service its own
|
||||
# environment, not the shell's locals — an unexported variable here goes
|
||||
# nowhere.
|
||||
#
|
||||
# Overridable, so a bad boot can be re-run with more without a rebuild.
|
||||
RUST_LOG="${RUST_LOG:-neshub=info}"
|
||||
export RUST_LOG
|
||||
30
build/etc/conf.d/nestri-user-env
Normal file
30
build/etc/conf.d/nestri-user-env
Normal file
@@ -0,0 +1,30 @@
|
||||
HOME="/home/nestri"
|
||||
USER="nestri"
|
||||
LOGNAME="nestri"
|
||||
XDG_RUNTIME_DIR="/run/user/1000"
|
||||
XDG_CONFIG_HOME="/home/nestri/.config"
|
||||
XDG_DATA_HOME="/home/nestri/.local/share"
|
||||
XDG_CACHE_HOME="/home/nestri/.cache"
|
||||
XDG_STATE_HOME="/home/nestri/.local/state"
|
||||
XDG_DATA_DIRS="/usr/local/share:/usr/share"
|
||||
XDG_SESSION_TYPE="wayland"
|
||||
XDG_SESSION_DESKTOP="nestri"
|
||||
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"
|
||||
|
||||
# Driver forcing, mirrored from /etc/profile.d/nestri-env.sh. That file only
|
||||
# runs for login shells, and nothing here is a login shell — every service
|
||||
# below runs from OpenRC. Capture happens through a Vulkan layer, so a
|
||||
# process that reached the GPU via native OpenGL would render fine and
|
||||
# capture nothing; zink is what makes such a process capturable at all.
|
||||
# These are load-bearing. Keep them in step with the profile.d copy.
|
||||
#__GLX_VENDOR_LIBRARY_NAME="mesa"
|
||||
#MESA_LOADER_DRIVER_OVERRIDE="zink"
|
||||
#GALLIUM_DRIVER="zink"
|
||||
|
||||
nestri_export_env() {
|
||||
export HOME USER LOGNAME
|
||||
export XDG_RUNTIME_DIR XDG_CONFIG_HOME XDG_DATA_HOME XDG_CACHE_HOME XDG_STATE_HOME
|
||||
export XDG_DATA_DIRS XDG_SESSION_TYPE XDG_SESSION_DESKTOP
|
||||
export DBUS_SESSION_BUS_ADDRESS
|
||||
#export __GLX_VENDOR_LIBRARY_NAME MESA_LOADER_DRIVER_OVERRIDE GALLIUM_DRIVER
|
||||
}
|
||||
32
build/etc/conf.d/neswire
Normal file
32
build/etc/conf.d/neswire
Normal file
@@ -0,0 +1,32 @@
|
||||
# Configuration for the audio wire.
|
||||
#
|
||||
# Not a symlink to nestri-user-env like pipewire and friends, for the same
|
||||
# reason neshub is not: it needs a setting of its own -- and it still wants
|
||||
# that file's environment, so it sources it rather than replacing it.
|
||||
. /etc/conf.d/nestri-user-env
|
||||
|
||||
# What neswire logs, and nothing else.
|
||||
#
|
||||
# Without this neswire is *silent* -- it starts, fails, and exits leaving an
|
||||
# empty log, which is indistinguishable from never having been started at
|
||||
# all. That ambiguity is the reason this file exists.
|
||||
#
|
||||
# Scoped, and `export`ed, for the reasons spelled out at length in
|
||||
# conf.d/neshub: an unscoped `info` turns on every linked crate, and an
|
||||
# unexported one reaches the service's shell and not the service.
|
||||
#
|
||||
# Overridable, so a bad boot can be re-run with more without a rebuild.
|
||||
RUST_LOG="${RUST_LOG:-neswire=info}"
|
||||
export RUST_LOG
|
||||
|
||||
# neswire's own defaults, spelled out rather than inherited.
|
||||
#
|
||||
# All four are clap args with defaults (`neswire --help`), so the service
|
||||
# works without them. They are here because the IPC path is a contract with
|
||||
# neshub -- it reads this socket -- and a contract that lives only inside two
|
||||
# binaries' default values is one nobody can check.
|
||||
NESWIRE_IPC_PATH="/tmp/nestri-audio.sock"
|
||||
NESWIRE_CHANNELS="2"
|
||||
NESWIRE_PACKET_DURATION_MS="5"
|
||||
NESWIRE_BITRATE_PER_CHANNEL="64"
|
||||
export NESWIRE_IPC_PATH NESWIRE_CHANNELS NESWIRE_PACKET_DURATION_MS NESWIRE_BITRATE_PER_CHANNEL
|
||||
29
build/etc/fstab
Normal file
29
build/etc/fstab
Normal file
@@ -0,0 +1,29 @@
|
||||
/dev/vda / ext4 rw,relatime 0 1
|
||||
devtmpfs /dev devtmpfs rw,nosuid 0 0
|
||||
proc /proc proc rw,nosuid,nodev,noexec 0 0
|
||||
sysfs /sys sysfs rw,nosuid,nodev,noexec 0 0
|
||||
tmpfs /tmp tmpfs rw,nosuid,nodev,size=64M 0 0
|
||||
tmpfs /run tmpfs rw,nosuid,nodev,size=32M,mode=0755 0 0
|
||||
tmpfs /var/log tmpfs rw,nosuid,nodev,size=16M 0 0
|
||||
# POSIX shared memory. devtmpfs does not provide it and nothing else mounts it,
|
||||
# so without this entry /dev/shm does not exist at all and `shm_open` fails.
|
||||
#
|
||||
# PipeWire itself gets by: it allocates buffers with memfd_create and only
|
||||
# falls back to /dev/shm. Kept as a precaution for anything else that uses
|
||||
# POSIX shm/semaphores directly — Proton is the known example, and it is not
|
||||
# part of this image yet (see build/README.md), but a failure here is silent
|
||||
# rather than fatal, so it costs nothing to have ready.
|
||||
#
|
||||
# `nosuid,nodev` and a size cap because everything a game can write to should
|
||||
# have both.
|
||||
tmpfs /dev/shm tmpfs rw,nosuid,nodev,size=256M 0 0
|
||||
# The guest's logs, on a host directory that outlives the VM.
|
||||
#
|
||||
# Here rather than mounted by a service, because the failures worth reading
|
||||
# are the ones that happen *before* anything else has mounted something — a
|
||||
# log directory that appears only after a successful start cannot record an
|
||||
# unsuccessful one. The tag is fixed by nessh, so plain fstab works.
|
||||
#
|
||||
# `nofail` because a VM started without the share still has to boot: that is
|
||||
# how somebody gets a shell to find out why it has no share.
|
||||
logs /nestri/logs virtiofs rw,nofail 0 0
|
||||
33
build/etc/init.d/dbus-session
Normal file
33
build/etc/init.d/dbus-session
Normal file
@@ -0,0 +1,33 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="D-Bus session bus for nestri user"
|
||||
|
||||
nestri_export_env
|
||||
|
||||
command="/usr/bin/dbus-daemon"
|
||||
command_args="--session --address=unix:path=/run/user/1000/bus --nofork --nopidfile --print-address"
|
||||
command_user="nestri:nestri"
|
||||
command_background="yes"
|
||||
pidfile="/run/nestri/dbus-session.pid"
|
||||
|
||||
depend() {
|
||||
need xdg-runtime dbus
|
||||
before pipewire wireplumber
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
checkpath -d -m 0755 -o "nestri:nestri" /run/nestri
|
||||
if [ -e "/run/user/1000/bus" ]; then
|
||||
ewarn "Removing stale bus socket"
|
||||
rm -f "/run/user/1000/bus"
|
||||
fi
|
||||
}
|
||||
|
||||
start_post() {
|
||||
local i=0
|
||||
while [ ! -S /run/user/1000/bus ] && [ $i -lt 20 ]; do
|
||||
sleep 0.1
|
||||
i=$((i+1))
|
||||
done
|
||||
[ -S /run/user/1000/bus ] || { eerror "Bus didn't appear"; return 1; }
|
||||
}
|
||||
19
build/etc/init.d/dbus-system
Normal file
19
build/etc/init.d/dbus-system
Normal file
@@ -0,0 +1,19 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="D-Bus system message bus"
|
||||
|
||||
command="/usr/bin/dbus-daemon"
|
||||
command_args="--system --nofork --nopidfile"
|
||||
command_background="yes"
|
||||
pidfile="/run/dbus/dbus.pid"
|
||||
|
||||
depend() {
|
||||
need localmount xdg-runtime
|
||||
before pipewire
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
checkpath -d -m 0755 -o root:root /run/dbus
|
||||
checkpath -d -m 0755 -o messagebus:messagebus /var/run/dbus 2>/dev/null || true
|
||||
dbus-uuidgen --ensure=/var/lib/dbus/machine-id
|
||||
}
|
||||
92
build/etc/init.d/guest-net
Normal file
92
build/etc/init.d/guest-net
Normal file
@@ -0,0 +1,92 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="Configure the guest's network to match the host's tap"
|
||||
|
||||
# Overridden from /etc/conf.d/guest-net if present. These defaults match
|
||||
# nesbox's own defaults; if you change the `network` section in the VM's
|
||||
# JSON, change these to match.
|
||||
: ${GUEST_IFACE:=eth0}
|
||||
: ${GUEST_IP:=172.30.0.2}
|
||||
: ${GUEST_PREFIX:=24}
|
||||
: ${GUEST_GATEWAY:=172.30.0.1}
|
||||
|
||||
depend() {
|
||||
need localmount
|
||||
provide net
|
||||
keyword -shutdown
|
||||
}
|
||||
|
||||
# Read one `nestri.<key>=<value>` from the kernel command line.
|
||||
#
|
||||
# The address has to come from somewhere per-boot, because the alternative --
|
||||
# baking it into the image -- makes every guest built from that image the
|
||||
# same host on the network. Two sandboxes then collide the moment they run
|
||||
# together.
|
||||
#
|
||||
# `nestri.`-prefixed rather than the kernel's own `ip=`: that one needs
|
||||
# CONFIG_IP_PNP and exists to configure NFS root, and the prefix makes it
|
||||
# obvious whose parameter this is.
|
||||
cmdline_value() {
|
||||
local key="$1" word
|
||||
for word in $(cat /proc/cmdline 2>/dev/null); do
|
||||
case "$word" in
|
||||
"nestri.${key}="*) printf '%s' "${word#nestri.${key}=}"; return 0 ;;
|
||||
esac
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
start() {
|
||||
ebegin "Bringing up loopback"
|
||||
ip link set lo up
|
||||
eend $?
|
||||
|
||||
# The command line wins over conf.d when it says anything, and conf.d is
|
||||
# the fallback so a hand-written VM config with no parameters keeps
|
||||
# working -- which is how a guest gets debugged.
|
||||
local source="/etc/conf.d/guest-net"
|
||||
local cmdline_ip
|
||||
if cmdline_ip="$(cmdline_value ip)"; then
|
||||
# Accepts address/prefix; a bare address keeps the configured prefix
|
||||
# rather than guessing one.
|
||||
case "$cmdline_ip" in
|
||||
*/*)
|
||||
GUEST_IP="${cmdline_ip%%/*}"
|
||||
GUEST_PREFIX="${cmdline_ip##*/}"
|
||||
;;
|
||||
*) GUEST_IP="$cmdline_ip" ;;
|
||||
esac
|
||||
source="kernel command line"
|
||||
fi
|
||||
|
||||
local cmdline_gw
|
||||
if cmdline_gw="$(cmdline_value gw)"; then
|
||||
GUEST_GATEWAY="$cmdline_gw"
|
||||
source="kernel command line"
|
||||
fi
|
||||
|
||||
# The VM may have been started with no network device at all, which is a
|
||||
# perfectly good configuration. Do not fail the boot over it.
|
||||
if [ ! -e "/sys/class/net/${GUEST_IFACE}" ]; then
|
||||
einfo "no ${GUEST_IFACE}: this VM has no network device"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Says which source won, because "the address is wrong" and "the address
|
||||
# came from somewhere unexpected" look identical from inside the guest.
|
||||
ebegin "Configuring ${GUEST_IFACE} as ${GUEST_IP}/${GUEST_PREFIX} via ${GUEST_GATEWAY} (from ${source})"
|
||||
# `replace` rather than `add` so a restart is not an error.
|
||||
ip link set "${GUEST_IFACE}" up &&
|
||||
ip addr replace "${GUEST_IP}/${GUEST_PREFIX}" dev "${GUEST_IFACE}" &&
|
||||
ip route replace default via "${GUEST_GATEWAY}" dev "${GUEST_IFACE}"
|
||||
eend $? "could not configure ${GUEST_IFACE}"
|
||||
}
|
||||
|
||||
stop() {
|
||||
if [ -e "/sys/class/net/${GUEST_IFACE}" ]; then
|
||||
ebegin "Bringing down ${GUEST_IFACE}"
|
||||
ip link set "${GUEST_IFACE}" down
|
||||
eend 0
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
33
build/etc/init.d/nescope
Normal file
33
build/etc/init.d/nescope
Normal file
@@ -0,0 +1,33 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="Nestri headless compositor"
|
||||
|
||||
nestri_export_env
|
||||
|
||||
: "${NESTRI_UID:=1000}"
|
||||
: "${NESTRI_USER:=nestri}"
|
||||
|
||||
command="/usr/bin/nescope"
|
||||
# No command after `--`: plain-compositor mode. nescope comes up and waits
|
||||
# for something to connect rather than wrapping a payload — starting one is
|
||||
# nesinit's job, and nesinit is not open code yet. See build/README.md.
|
||||
command_user="${NESTRI_USER}:${NESTRI_USER}"
|
||||
command_background="yes"
|
||||
pidfile="/run/nestri/nescope.pid"
|
||||
output_log="/nestri/logs/nescope.log"
|
||||
error_log="/nestri/logs/nescope.log"
|
||||
respawn="yes"
|
||||
respawn_delay="2"
|
||||
respawn_max="2"
|
||||
|
||||
export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}"
|
||||
|
||||
depend() {
|
||||
need xdg-runtime
|
||||
use neshub
|
||||
after xdg-runtime
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/nestri
|
||||
}
|
||||
39
build/etc/init.d/neshub
Normal file
39
build/etc/init.d/neshub
Normal file
@@ -0,0 +1,39 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="Nestri media hub"
|
||||
|
||||
nestri_export_env
|
||||
|
||||
: "${NESTRI_UID:=1000}"
|
||||
: "${NESTRI_USER:=nestri}"
|
||||
|
||||
command="/usr/bin/neshub"
|
||||
# Unprivileged. The old nestri-guest-hub ran as root and mounted the
|
||||
# session's filesystem itself; the open neshub does neither — per its own
|
||||
# README it only muxes the Unix sockets the other components dial into one
|
||||
# QUIC endpoint. Whatever ends up owning session mounts (nesinit, presumably)
|
||||
# is a separate, still-closed piece.
|
||||
command_user="${NESTRI_USER}:${NESTRI_USER}"
|
||||
command_background="yes"
|
||||
pidfile="/run/nestri/neshub.pid"
|
||||
|
||||
# Where neshub's output goes. /nestri/logs is a virtiofs share mounted from
|
||||
# /etc/fstab at boot, before this service starts, so a hub that fails
|
||||
# immediately still leaves a record, and the record survives the VM.
|
||||
output_log="/nestri/logs/neshub.log"
|
||||
error_log="/nestri/logs/neshub.log"
|
||||
respawn="yes"
|
||||
respawn_delay="2"
|
||||
respawn_max="2" # NO infinite — if it fails, it fails for a reason
|
||||
|
||||
export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}"
|
||||
|
||||
depend() {
|
||||
need xdg-runtime
|
||||
use net
|
||||
after xdg-runtime
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/nestri
|
||||
}
|
||||
93
build/etc/init.d/neswire
Normal file
93
build/etc/init.d/neswire
Normal file
@@ -0,0 +1,93 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="Nestri pipewire audio sink"
|
||||
|
||||
nestri_export_env
|
||||
|
||||
: "${NESTRI_UID:=1000}"
|
||||
: "${NESTRI_USER:=nestri}"
|
||||
|
||||
command="/usr/bin/neswire"
|
||||
command_user="${NESTRI_USER}:${NESTRI_USER}"
|
||||
command_background="yes"
|
||||
pidfile="/run/nestri/neswire.pid"
|
||||
output_log="/nestri/logs/neswire.log"
|
||||
error_log="/nestri/logs/neswire.log"
|
||||
respawn="yes"
|
||||
respawn_delay="2"
|
||||
respawn_max="2"
|
||||
|
||||
export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}"
|
||||
export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/${NESTRI_UID}/bus"
|
||||
|
||||
depend() {
|
||||
# wireplumber is a hard dependency, not a nicety: start_post below pins
|
||||
# the default sink through the `default` metadata object, and
|
||||
# WirePlumber is what owns that object. Started concurrently, the pin
|
||||
# lands in an object pw-metadata created itself, which is destroyed the
|
||||
# moment it exits.
|
||||
need xdg-runtime dbus dbus-session pipewire wireplumber
|
||||
use neshub
|
||||
after pipewire wireplumber neshub
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/nestri
|
||||
}
|
||||
|
||||
# Point the graph at neswire's sink, by name.
|
||||
#
|
||||
# `neswire` registers itself as media.class = Audio/Sink, node.name = neswire
|
||||
# ("Neswire Cloud Gaming Audio Sink"). With the hardware monitors off it is
|
||||
# the only sink, so WirePlumber's find-best hook should land on it anyway --
|
||||
# this makes it explicit rather than a consequence of there being nothing
|
||||
# else, so adding a second sink later cannot silently steal the default.
|
||||
#
|
||||
# `default.configured.audio.sink` is the metadata WirePlumber's find-selected
|
||||
# hook reads, and it takes a name; `wpctl set-default` takes a numeric object
|
||||
# id that changes every boot, which is why this uses pw-metadata instead.
|
||||
#
|
||||
# Not fatal if it fails: find-best still has one candidate. A game playing
|
||||
# into the wrong sink is silent, and a warning here is the only place that
|
||||
# would say so.
|
||||
start_post() {
|
||||
local i=0
|
||||
while [ $i -lt 50 ]; do
|
||||
pw-dump 2>/dev/null | grep -q '"node.name": "neswire"' && break
|
||||
sleep 0.1
|
||||
i=$((i+1))
|
||||
done
|
||||
if ! pw-dump 2>/dev/null | grep -q '"node.name": "neswire"'; then
|
||||
ewarn "neswire started but its sink never appeared in the graph"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Wait for WirePlumber to own the `default` metadata before writing to it.
|
||||
#
|
||||
# `need wireplumber` only guarantees its script returned, not that it has
|
||||
# built its objects. Writing too early is silently useless rather than an
|
||||
# error: pw-metadata creates the object, sets the key, exits 0, and the
|
||||
# object dies with the client. So wait for the object to exist, and say
|
||||
# so if it never does.
|
||||
i=0
|
||||
while [ $i -lt 50 ]; do
|
||||
pw-metadata -n default >/dev/null 2>&1 && break
|
||||
sleep 0.1
|
||||
i=$((i+1))
|
||||
done
|
||||
if ! pw-metadata -n default >/dev/null 2>&1; then
|
||||
ewarn "no 'default' metadata: WirePlumber is not running, sink not pinned"
|
||||
return 0
|
||||
fi
|
||||
|
||||
pw-metadata -n default 0 default.configured.audio.sink '{ "name": "neswire" }' \
|
||||
>/dev/null 2>&1
|
||||
|
||||
# Read it back. A write that did not stick is the failure this whole
|
||||
# sequence exists to catch, and it is invisible unless checked.
|
||||
if pw-metadata -n default 2>/dev/null | grep -q "neswire"; then
|
||||
einfo "default sink pinned to neswire"
|
||||
else
|
||||
ewarn "pinned neswire as default sink but it did not stick"
|
||||
fi
|
||||
}
|
||||
45
build/etc/init.d/pipewire
Normal file
45
build/etc/init.d/pipewire
Normal file
@@ -0,0 +1,45 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="PipeWire multimedia daemon (system mode for nestri)"
|
||||
|
||||
nestri_export_env
|
||||
|
||||
: "${NESTRI_UID:=1000}"
|
||||
: "${NESTRI_USER:=nestri}"
|
||||
|
||||
command="/usr/bin/pipewire"
|
||||
command_user="${NESTRI_USER}:${NESTRI_USER}"
|
||||
command_background="yes"
|
||||
pidfile="/run/pipewire/pipewire.pid"
|
||||
|
||||
export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}"
|
||||
export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/${NESTRI_UID}/bus"
|
||||
|
||||
depend() {
|
||||
need xdg-runtime dbus
|
||||
use dbus
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
checkpath -d -m 0755 -o "${NESTRI_USER}:${NESTRI_USER}" /run/pipewire
|
||||
}
|
||||
|
||||
# `need pipewire` only waits for this script to return, and with
|
||||
# command_background that is the moment start-stop-daemon forks -- not the
|
||||
# moment PipeWire is accepting connections. With rc_parallel="YES" every
|
||||
# client (wireplumber, neswire) then races the socket and a client that
|
||||
# loses simply exits.
|
||||
#
|
||||
# dbus-session already solved exactly this for the bus socket. Same shape
|
||||
# here.
|
||||
start_post() {
|
||||
local i=0
|
||||
while [ ! -S "/run/user/${NESTRI_UID}/pipewire-0" ] && [ $i -lt 50 ]; do
|
||||
sleep 0.1
|
||||
i=$((i+1))
|
||||
done
|
||||
[ -S "/run/user/${NESTRI_UID}/pipewire-0" ] || {
|
||||
eerror "PipeWire socket never appeared"
|
||||
return 1
|
||||
}
|
||||
}
|
||||
20
build/etc/init.d/wireplumber
Normal file
20
build/etc/init.d/wireplumber
Normal file
@@ -0,0 +1,20 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="WirePlumber session manager for PipeWire"
|
||||
|
||||
nestri_export_env
|
||||
|
||||
: "${NESTRI_UID:=1000}"
|
||||
: "${NESTRI_USER:=nestri}"
|
||||
|
||||
command="/usr/bin/wireplumber"
|
||||
command_user="${NESTRI_USER}:${NESTRI_USER}"
|
||||
command_background="yes"
|
||||
pidfile="/run/pipewire/wireplumber.pid"
|
||||
|
||||
export XDG_RUNTIME_DIR="/run/user/${NESTRI_UID}"
|
||||
|
||||
depend() {
|
||||
need pipewire
|
||||
after pipewire
|
||||
}
|
||||
44
build/etc/init.d/xdg-runtime
Normal file
44
build/etc/init.d/xdg-runtime
Normal file
@@ -0,0 +1,44 @@
|
||||
#!/sbin/openrc-run
|
||||
|
||||
description="Create runtime directories (XDG_RUNTIME_DIR, X11 socket dir..)"
|
||||
|
||||
NESTRI_UID="${NESTRI_UID:-1000}"
|
||||
NESTRI_USER="${NESTRI_USER:-nestri}"
|
||||
|
||||
depend() {
|
||||
need localmount
|
||||
before dbus pipewire
|
||||
}
|
||||
|
||||
start() {
|
||||
ebegin "Preparing runtime directories"
|
||||
|
||||
if ! getent passwd "${NESTRI_USER}" >/dev/null 2>&1; then
|
||||
eerror "User ${NESTRI_USER} does not exist"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Per-user XDG runtime dir
|
||||
mkdir -p "/run/user/${NESTRI_UID}"
|
||||
chown "${NESTRI_USER}:${NESTRI_USER}" "/run/user/${NESTRI_UID}"
|
||||
chmod 0700 "/run/user/${NESTRI_UID}"
|
||||
|
||||
# X11 socket dir (Xwayland + any X clients expect this)
|
||||
mkdir -p /tmp/.X11-unix
|
||||
chown root:root /tmp/.X11-unix
|
||||
chmod 1777 /tmp/.X11-unix
|
||||
|
||||
# ICE socket dir - some toolkits look for it
|
||||
mkdir -p /tmp/.ICE-unix
|
||||
chown root:root /tmp/.ICE-unix
|
||||
chmod 1777 /tmp/.ICE-unix
|
||||
|
||||
eend $?
|
||||
}
|
||||
|
||||
stop() {
|
||||
ebegin "Removing runtime directories"
|
||||
rm -rf "/run/user/${NESTRI_UID}"
|
||||
# Don't rm /tmp/.X11-unix on stop - other things may be using it
|
||||
eend 0
|
||||
}
|
||||
27
build/etc/profile.d/nestri-env.sh
Normal file
27
build/etc/profile.d/nestri-env.sh
Normal file
@@ -0,0 +1,27 @@
|
||||
# Nestri user environment — sourced for nestri user logins and su -
|
||||
if [ "$(id -u)" = "1000" ]; then
|
||||
export XDG_RUNTIME_DIR="/run/user/1000"
|
||||
export XDG_CONFIG_HOME="${HOME}/.config"
|
||||
export XDG_DATA_HOME="${HOME}/.local/share"
|
||||
export XDG_CACHE_HOME="${HOME}/.cache"
|
||||
export XDG_STATE_HOME="${HOME}/.local/state"
|
||||
|
||||
export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"
|
||||
|
||||
# Portal backend selection. "GTK" matches xdg-desktop-portal-gtk.
|
||||
# Once nescope supports wlr portals, change to "wlroots" or similar.
|
||||
#export XDG_CURRENT_DESKTOP="${XDG_CURRENT_DESKTOP:-GTK}"
|
||||
export XDG_SESSION_TYPE="${XDG_SESSION_TYPE:-wayland}"
|
||||
export XDG_SESSION_DESKTOP="${XDG_SESSION_DESKTOP:-nestri}"
|
||||
|
||||
# Ensure proper VAAPI driver is used
|
||||
#export LIBVA_DRIVER_NAME="radeonsi"
|
||||
|
||||
# Force zink usage for OpenGL -> Vulkan translation
|
||||
#export __GLX_VENDOR_LIBRARY_NAME=mesa
|
||||
#export MESA_LOADER_DRIVER_OVERRIDE=zink
|
||||
#export GALLIUM_DRIVER=zink
|
||||
|
||||
# Ensure standard XDG dirs exist
|
||||
mkdir -p "${XDG_CONFIG_HOME}" "${XDG_DATA_HOME}" "${XDG_CACHE_HOME}" "${XDG_STATE_HOME}" 2>/dev/null || true
|
||||
fi
|
||||
27
build/etc/rc.conf
Normal file
27
build/etc/rc.conf
Normal file
@@ -0,0 +1,27 @@
|
||||
# Nestri rootfs OpenRC config
|
||||
|
||||
rc_parallel="YES"
|
||||
rc_depend_strict="NO"
|
||||
rc_interactive="NO"
|
||||
rc_shell_timeout="0"
|
||||
# Every service's start/fail line, on the host directory that outlives the VM.
|
||||
#
|
||||
# Without this a guest nobody can log into reports nothing about its own boot:
|
||||
# a service that was skipped because a dependency failed looks exactly like a
|
||||
# service that was never registered. /nestri/logs is a virtiofs share, so the
|
||||
# record survives the guest — see the fstab entry below.
|
||||
rc_logger="YES"
|
||||
rc_log_path="/nestri/logs/rc.log"
|
||||
rc_verbose="NO"
|
||||
|
||||
# cgroup v2
|
||||
rc_cgroup_mode="unified"
|
||||
|
||||
# Don't bother trying to set hostname twice
|
||||
rc_hotplug="!net.*"
|
||||
|
||||
# Faster sulogin behavior on emergency
|
||||
rc_shell="/bin/sh"
|
||||
|
||||
# Default umask
|
||||
umask 022
|
||||
2
build/etc/resolv.conf
Normal file
2
build/etc/resolv.conf
Normal file
@@ -0,0 +1,2 @@
|
||||
nameserver 1.1.1.1
|
||||
nameserver 8.8.8.8
|
||||
41
build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf
Normal file
41
build/etc/wireplumber/wireplumber.conf.d/50-nestri.conf
Normal file
@@ -0,0 +1,41 @@
|
||||
# WirePlumber, trimmed to what a microVM with no sound card actually needs.
|
||||
#
|
||||
# The goal is that `neswire` is the *only* Audio/Sink in the graph, so
|
||||
# default-node selection has exactly one candidate and cannot pick wrong.
|
||||
#
|
||||
# The "Dummy Output" (`auto_null`) sink is not disabled here, and no longer
|
||||
# needs to be: it came from pipewire-pulse's `module-always-sink`, and
|
||||
# pipewire-pulse is no longer installed. WirePlumber ships
|
||||
# scripts/fallback-sink.lua, which creates a node by the same name, but no
|
||||
# component in wireplumber.conf references it, so it never loads. If auto_null
|
||||
# ever comes back, it came back with pipewire-pulse -- the switch is
|
||||
# `pulse.cmd.always-sink = false` in a pipewire-pulse.conf.d drop-in, not
|
||||
# anything on this side.
|
||||
#
|
||||
# pipewire.conf's Dummy-Driver / Freewheel-Driver are a third thing again:
|
||||
# support.node.driver objects with no ports -- drivers, not sinks, so nothing
|
||||
# can play into them. Dummy-Driver is load-bearing here. With no ALSA hardware
|
||||
# it is what clocks the graph, and therefore what paces neswire's sink. Do not
|
||||
# disable it.
|
||||
|
||||
wireplumber.profiles = {
|
||||
main = {
|
||||
# No sound card, no bluetooth adapter, no camera. These only exist to
|
||||
# probe hardware this guest does not have; disabling them removes the
|
||||
# only other source of sinks besides neswire.
|
||||
hardware.audio = disabled
|
||||
hardware.bluetooth = disabled
|
||||
hardware.video-capture = disabled
|
||||
|
||||
# The guest root is read-only and every VM is built from the same image, so
|
||||
# there is nothing to restore and nowhere to save. Left enabled, a stale
|
||||
# state file is a second, invisible place that can pin a default sink --
|
||||
# the exact failure this file exists to rule out.
|
||||
#
|
||||
# Equivalent to inheriting mixin.stateless, spelled out so it is greppable.
|
||||
hooks.default-nodes.state = disabled
|
||||
hooks.device.profile.state = disabled
|
||||
hooks.device.routes.state = disabled
|
||||
hooks.stream.state = disabled
|
||||
}
|
||||
}
|
||||
67
build/scripts/mkimage.sh
Normal file
67
build/scripts/mkimage.sh
Normal file
@@ -0,0 +1,67 @@
|
||||
#!/usr/bin/env bash
|
||||
# Pack a built container image into a raw ext4 disk for nesbox's virtio-blk.
|
||||
#
|
||||
# Docker/Buildx has no native "export a raw disk image" step, so this is the
|
||||
# one part of the pipeline that still has to run outside the Dockerfile.
|
||||
#
|
||||
# Run this as yourself, not under sudo. Only mkfs/mount/umount actually need
|
||||
# root, and those are escalated individually below — `sudo bash mkimage.sh`
|
||||
# for the whole script is exactly the wrong shape for rootless Podman: the
|
||||
# image `make build` produced lives in *your* rootless storage, and running
|
||||
# `podman create` as root afterwards looks in root's separate storage, where
|
||||
# the tag does not exist. `sudo -v` up front just avoids being prompted
|
||||
# mid-script for the escalated calls that follow.
|
||||
set -euo pipefail
|
||||
|
||||
IMAGE="${1:?usage: mkimage.sh <image-tag> <output-path> [size]}"
|
||||
OUT="${2:?usage: mkimage.sh <image-tag> <output-path> [size]}"
|
||||
SIZE="${3:-5G}"
|
||||
|
||||
if [[ "$(id -u)" -eq 0 ]]; then
|
||||
echo "mkimage.sh should run as yourself, not root/sudo — see the comment at the top of this script" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
CONTAINER_RT="$(command -v docker || command -v podman || true)"
|
||||
[[ -n "$CONTAINER_RT" ]] || { echo "Neither docker nor podman found in PATH" >&2; exit 1; }
|
||||
|
||||
sudo -v # cache credentials once, rather than prompting mid-pipeline
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
cleanup() {
|
||||
mountpoint -q "$WORK/mnt" 2>/dev/null && sudo umount "$WORK/mnt"
|
||||
rm -rf "$WORK"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
echo "Exporting ${IMAGE}..."
|
||||
cid="$("$CONTAINER_RT" create "$IMAGE")"
|
||||
"$CONTAINER_RT" export "$cid" -o "$WORK/rootfs.tar"
|
||||
"$CONTAINER_RT" rm -f "$cid" >/dev/null
|
||||
|
||||
echo "Creating ${SIZE} ext4 image at ${OUT}..."
|
||||
mkdir -p "$(dirname "$OUT")"
|
||||
truncate -s "$SIZE" "$OUT"
|
||||
sudo mkfs.ext4 -q -L nestri-root "$OUT"
|
||||
|
||||
mkdir -p "$WORK/mnt"
|
||||
sudo mount -o loop "$OUT" "$WORK/mnt"
|
||||
# Same excludes as the old bootstrap extraction: .dockerenv is Docker's own
|
||||
# marker file, and /dev is devtmpfs at boot, populated by the kernel — a
|
||||
# tarred copy of the build container's /dev would just be dead weight.
|
||||
#
|
||||
# Root, deliberately: the rootfs's own files are owned by root (that is
|
||||
# correct — it is the guest's root filesystem), and only root can write
|
||||
# root-owned files onto the loop-mounted ext4.
|
||||
sudo tar -xf "$WORK/rootfs.tar" -C "$WORK/mnt" --exclude='.dockerenv' --exclude='dev/*'
|
||||
sudo umount "$WORK/mnt"
|
||||
|
||||
# The image *file* itself was created by `truncate` above as the invoking
|
||||
# user and never needs to change hands — mkfs/mount/umount touch its
|
||||
# contents, not its ownership. Asserted, not assumed, since a stray `sudo`
|
||||
# reordering above would silently hand root ownership of a file the rest of
|
||||
# this pipeline expects to read and delete without sudo.
|
||||
[[ "$(stat -c '%U' "$OUT")" == "$(id -un)" ]] || {
|
||||
echo "warning: ${OUT} is not owned by $(id -un) — sudo chown $(id -un) ${OUT}" >&2
|
||||
}
|
||||
echo "Wrote ${OUT}"
|
||||
Reference in New Issue
Block a user