github-mirrors/netris-nestri
1
0
Fork 0
mirror of https://github.com/nestriness/nestri.git synced 2026-09-19 09:15:19 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: resident guest init (#333)

Browse Source 
Get this thing going..







<!-- greptile_comment -->

<!-- greptile_summary -->

<h2><a
href="https://app.greptile.com/api/retrigger?id=63134761"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/RetriggerDark.svg?v=1"><source
media="(prefers-color-scheme: light)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1"><img
alt="Retrigger"
src="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1"
align="right"></picture></a>Confidence Score: 5/5</h2>

The PR appears safe to merge; all previous findings are resolved and the
latest readiness change introduces no established actionable regression.

<h3>Summary</h3>

- Establishes required guest filesystems, runtime directories, device
permissions, and service processes.
- Reports initialization and service deaths over the lifecycle channel.
- Supports launch, restart, and shutdown commands for a resident guest.
- Separates service and workload identities and configures per-launch
runtime environments.
- Removes the currently inactive nescope screenshot option and makes
capture-chain verification fail explicitly when compositor readback is
unavailable.
- Reworks the guest image around `nesinit` as PID 1 without a
distribution service manager.

<h3>Diagram</h3>

```mermaid
sequenceDiagram
    participant Host
    participant Init as nesinit
    participant FS as Guest filesystems
    participant Services as Service stack
    participant Workload

    Init->>Host: Ready(protocol version)
    Host->>Init: Boot(mount descriptors)
    Init->>FS: Establish and mount shares
    Init->>Services: Spawn services in order
    Services-->>Init: Required sockets ready
    Init->>Host: Initialized(service names)
    Host->>Init: Launch(id, exec, on_exit)
    Init->>Workload: Spawn with isolated UID/runtime
    Init->>Host: Started(id)
    Workload-->>Init: Exit status
    Init->>Host: WorkloadExited(id, status)
    Host->>Init: Launch / Restart / Shutdown
```

<sub>Reviews (4) · Last reviewed commit: ["fix(nesinit): readiness is a
socket
that..."](731d34df9d)</sub>

<!-- /greptile_comment -->

---------

Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kristian Ollikainen
2026-09-14 14:45:13 +03:00
committed by GitHub
parent ec8b13d0c9
commit 8246aa5538
43 changed files with 4448 additions and 1553 deletions
Download Patch File Download Diff File Expand all files Collapse all files

75
apps/nesinit/README.md
View File

@@ -2,8 +2,9 @@
PID 1 inside a box.
A microVM has no init unless something is it. Three of the jobs are nobody
else's, and this is all of them:
A microVM has no init unless something is it, and in a box nothing else is:
there is no service manager in the image and no init scripts. Four jobs, and
this is all of them:
- **Reaping.** A process whose parent dies is reparented to PID 1. Without a
reaper, every orphan the workload leaves behind holds a pid and a slot in the
@@ -11,25 +12,46 @@ else's, and this is all of them:
- **Ordered shutdown.** The workload stops first and alone, then everything
else, then the disks are flushed and the machine is powered off. An init that
returns leaves a guest running with nothing in it.
- **The box's own services.** The bus, audio, and the transport that carries a
session out, started in order from a table compiled into this binary. There
is no unit format and no directory of files to read: the services in a box
are fixed, and running on any distribution comes from depending on none of
their init scripts rather than from being configurable.
- **The guest end of the control channel.** One vsock connection out, carrying
what to run in and what happened back.
It does not know what it is running. It is handed a command line, a set of
shares and what an exit means; there is no code path here that branches on
which workload it started, and there is not meant to be.
It does not know what it is running. It is handed a set of shares, and then
commands naming what to run and what an exit means; there is no code path here
that branches on which workload it started, and there is not meant to be.
**A box outlives what runs in it.** Init mounts, brings the services up, says
it is ready, and then takes commands for as long as the box lives — so this
image on its own runs nothing at all, and a box may be launched into more than
once.
### The channel
The guest dials out on a fixed vsock port and speaks first:
```
guest → { "type": "ready", "protocol_version": 2 }
guest ← { "type": "boot", "exec": {...}, "mounts": [...], "geometry": {...}, "on_exit": {...} }
guest → { "type": "ready", "protocol_version": 3 }
guest ← { "type": "boot", "mounts": [...] }
guest → { "type": "mounted" }
guest → { "type": "started" }
guest → { "type": "workload_exited", "exit_code": 0 }
guest → { "type": "initialized", "services": ["dbus-system", ...] }
guest ← { "type": "launch", "id": "…", "exec": {...}, "on_exit": {...} }
guest → { "type": "started", "id": "…" }
guest → { "type": "workload_exited", "id": "…", "exit_code": 0 }
```
`ready` is the handshake and `initialized` is the box working. They are two
facts and must not be treated as one: a caller that waits on the first has a
wait that succeeds before anything in the guest has started.
Every launch carries an id and every message about a launch carries it back.
Without one, a second launch's exit is indistinguishable from the first's —
which reads at the far end as a finished session still running, or a running
one reported as stopped.
Newline-delimited JSON. Dialling out rather than being connected to is worth
keeping for two reasons: the listener is up before the VM starts, so nothing
races a booting kernel and nothing has to retry, and the connection
@@ -109,19 +131,40 @@ later for no visible reason.
### It reports; it does not supervise
When the workload ends, the exit goes up the channel and the session is over.
`on_exit` says what that exit *means* — whether it ends the session — and
nothing here restarts anything. Starting something again is a decision for the
end that can see whether restarting is repair or a loop.
When a launch ends, the exit goes up the channel. `on_exit` says what that exit
*means* — whether it ends the session or leaves the box up to be launched into
again — and nothing here restarts anything of its own accord. Starting
something again is a decision for the end that can see whether restarting is
repair or a loop.
`restart` exists as one message and is defined as exactly that: a kill followed
by a launch of the same command, keeping the id, with no retry and no backoff.
It is one message rather than two only because a caller sending two has the
same effect with a worse race in it.
The same rule covers the box's own services. One that dies is **reported and
left dead** — nothing else in the guest is watching them, so a death that is
not said here is a box that looks healthy and cannot work.
**One launch at a time.** A launch arriving while one is running is refused,
carrying the id it was asked for, rather than queued or silently replacing it:
a box has one screen, so a second concurrent launch has nowhere to draw.
A signalled workload is reported as signalled, with no exit code. Reporting
`0` for a killed process would make a kill look like a clean run.
### What is not here yet
`geometry` is carried and parsed but nothing consumes it: nesinit does not
start the guest's own services yet. `ticket` exists as a message with no
producer wired to it.
**It has never been PID 1 of anything.** Every line of this is written for a
box and all of it has been tested outside one. It runs perfectly well as an
ordinary process — it warns rather than fails when it is not PID 1 — which is
how most of it is exercised, and is also how a guest that will not boot gets
debugged: `init=/bin/bash` on the kernel command line, then run this by hand
and watch it fail.
Output geometry is deliberately absent from this layer. The compositor is
started by a launch, with that launch's geometry in its own arguments, so the
numbers appear in one place rather than two that can disagree.
### Testing

158
apps/nesinit/src/filesystems.rs
View File

@@ -31,6 +31,12 @@ struct Early {
/// so a device node or a setuid bit appearing in one did not come from us.
const NOSUID_NODEV: libc::c_ulong = libc::MS_NOSUID | libc::MS_NODEV;
// Every tmpfs below is capped, and the caps are load-bearing rather than
// tidiness. A tmpfs with no `size=` may grow to half of RAM, and the RAM in
// question is the box's whole allowance — so an uncapped `/tmp` is a workload
// that can OOM the box it runs in by writing files. The numbers are carried
// over from the mount table this replaced, where they were already considered.
const EARLY: &[Early] = &[
Early {
source: "proc",
@@ -41,6 +47,38 @@ const EARLY: &[Early] = &[
cost: "this process cannot make itself ineligible for the OOM killer, \
and nothing in the guest can read its own state",
},
// Usually already there: a kernel built with `CONFIG_DEVTMPFS_MOUNT` mounts
// this before init runs. Listed anyway because the check below skips what
// is already mounted, so the entry costs nothing when the kernel did it and
// is the difference between a working box and one with no device nodes when
// it did not. Without `nodev`, obviously — device nodes are the point.
Early {
source: "devtmpfs",
target: "/dev",
fstype: "devtmpfs",
flags: libc::MS_NOSUID,
data: "mode=755",
cost: "there are no device nodes at all, so nothing can open the GPU",
},
Early {
source: "devpts",
target: "/dev/pts",
fstype: "devpts",
flags: NOSUID_NODEV | libc::MS_NOEXEC,
data: "mode=620,gid=5,ptmxmode=666",
cost: "nothing that wants a terminal can allocate one",
},
// The image creates this directory, and the mode is the load-bearing part:
// a workload and the box's own services are different users, and shared
// memory between them is how a Vulkan client hands buffers around.
Early {
source: "tmpfs",
target: "/dev/shm",
fstype: "tmpfs",
flags: NOSUID_NODEV,
data: "mode=1777,size=256m",
cost: "anything using shared memory fails, which includes most graphics",
},
Early {
source: "sysfs",
target: "/sys",
@@ -59,7 +97,7 @@ const EARLY: &[Early] = &[
flags: NOSUID_NODEV,
// The sticky bit, because the workload does not run as this process
// does and what it binds here is its own.
data: "mode=1777",
data: "mode=1777,size=64m",
cost: "whatever serves this session's address cannot bind its socket, \
so the session never gets one",
},
@@ -74,20 +112,51 @@ const EARLY: &[Early] = &[
// Octal, and without a leading zero on purpose: the kernel parses a
// tmpfs mode as octal either way, and this is the spelling `mount`
// itself documents.
data: "mode=755",
data: "mode=755,size=32m",
cost: "there is nowhere for a runtime socket to live, so neither the \
payload relay nor this session's address can be served",
},
// The tree a session's shares are mounted into.
//
// A share's target is named by the descriptor and may be any path under
// here, so something has to create directories on a root that is read-only
// by design. That is what this is: `workload::mount` calls `create_dir_all`
// on each target, which fails with `EROFS` unless the tree it is creating
// in is writable.
//
// **This entry used to be forbidden, and the reason it was forbidden is
// gone.** A test here asserted that `/nestri` must never be mounted over,
// because a fresh tmpfs would hide the install, the user state and the work
// directory that the image had prepared underneath. That was true of the
// image that shipped those directories and an `fstab` that mounted into
// them. The image prepares nothing here now — the host names every share
// and every target — so there is nothing left to hide, and the rule had
// become a guard on a hazard that was deleted with the image that had it.
// ref(d-0064)
//
// Small on purpose. Everything real is mounted *over* this, so what remains
// is a handful of empty directories; the cap matters for the case where a
// share fails to mount and a workload writes to the bare mount point
// instead, which would otherwise be RAM the box cannot get back.
Early {
source: "tmpfs",
target: "/nestri",
fstype: "tmpfs",
flags: NOSUID_NODEV,
data: "mode=755,size=4m",
cost: "no share can be mounted, because its target cannot be created on a read-only root",
},
// The relay's own directory, and it is deliberately **not** in the tree the
// session's shares live in.
//
// It was, and that was wrong in a way no test here would have caught: a
// fresh tmpfs over the share tree hides every directory the image prepared
// underneath it — the install, the user state, the work directory, and the
// mount point the log share is attached to from `fstab`. The box then has a
// socket and none of the places its workload expects to find its files, and
// the exact-path check below cannot notice, because what `fstab` mounts is
// a directory *inside* that tree rather than the tree itself.
// It was, and moving it out stays right for a reason that outlived the one
// originally given. The first reason was that a tmpfs over the share tree
// would hide what the image had prepared there; that image is gone and the
// entry above now mounts that tree deliberately. The reason that remains is
// ownership: this directory is written by this process and by nothing else,
// which is what makes the socket in it unreplaceable. The share tree is
// mounted into by the host's own shares, so a relay socket living there
// would sit in a tree a workload's own share can be attached over.
//
// Owned by this process and writable by nothing else, which is what makes
// the socket in it unreplaceable. The workload reaches it because the
@@ -98,10 +167,22 @@ const EARLY: &[Early] = &[
target: crate::payload::DIRECTORY,
fstype: "tmpfs",
flags: NOSUID_NODEV | libc::MS_NOEXEC,
data: "mode=755",
// One socket lives here, so this is as small as a tmpfs usefully gets.
data: "mode=755,size=1m",
cost: "the payload relay cannot bind, so nothing reaches the workload \
over the channel",
},
// The root is read-only and some things write here whether or not anything
// reads it back. A box's real logs leave over the control channel; this is
// so that a library writing a file does not fail on `EROFS` instead.
Early {
source: "tmpfs",
target: "/var/log",
fstype: "tmpfs",
flags: NOSUID_NODEV | libc::MS_NOEXEC,
data: "mode=755,size=16m",
cost: "anything that writes a log file fails on a read-only root",
},
];
/// Mount what the rest of this component assumes is already there.
@@ -220,6 +301,25 @@ mod tests {
}
}
/// **Every tmpfs is capped.** One without a `size=` may grow to half of RAM,
/// and the RAM in question is the whole box's — so an uncapped `/tmp` hands
/// a workload a way to OOM the box it is running in by writing files. The
/// failure looks like a box that died under load rather than like a missing
/// mount option, which is why this is a test.
#[test]
fn no_tmpfs_is_unbounded() {
for early in EARLY {
if early.fstype != "tmpfs" {
continue;
}
assert!(
early.data.contains("size="),
"{} is an uncapped tmpfs",
early.target
);
}
}
/// `/proc` is mounted before `mountinfo` is read, so it has to be first.
#[test]
fn proc_is_the_first_entry() {
@@ -243,20 +343,32 @@ mod tests {
}
/// **Nothing here may be mounted over the tree the session's shares live
/// in.** A fresh tmpfs there hides every directory the image prepared
/// underneath — the install, the user state, the work directory, and the
/// mount point the log share attaches to — and the exact-path check cannot
/// notice, because what is mounted from `fstab` is a directory inside that
/// tree rather than the tree itself. So a box would come up with a socket
/// and without any of the places its workload looks for its files.
/// in**, and the share tree itself is mounted so that targets under it can
/// be created at all.
///
/// This replaces a test that asserted the exact opposite — that `/nestri`
/// must never be mounted over — on the grounds that a tmpfs there would
/// hide the install, the user state and the work directory the image had
/// prepared. The image that prepared them no longer exists; the host names
/// every share and every target now, and a read-only root cannot have a
/// directory created on it. Measured 2026-09-11: without this entry the
/// first real boot refused its own descriptor with
/// `/nestri/payload: Read-only file system`.
#[test]
fn the_share_tree_is_never_mounted_over() {
for early in EARLY {
assert_ne!(
early.target, "/nestri",
"this hides the directories the image prepared for a session"
);
}
fn the_share_tree_is_writable_and_the_relay_is_not_inside_it() {
let tree = EARLY
.iter()
.find(|e| e.target == "/nestri")
.expect("a share's target cannot be created without this");
assert_eq!(tree.fstype, "tmpfs");
assert!(
tree.data.contains("size="),
"an uncapped tmpfs here is RAM a box cannot get back"
);
assert!(
!crate::payload::DIRECTORY.starts_with("/nestri/"),
"the relay's socket would sit in a tree a share can be mounted over"
);
}
/// The relay's directory is the one this cannot hardcode: it belongs to

12
apps/nesinit/src/lib.rs
View File

@@ -4,14 +4,20 @@
// nobody else's: reaping whatever the workload orphans, turning a signal into
// an ordered shutdown, and being the guest end of the one channel out.
//
// It does not know what it is running. It is handed a command, a set of shares
// and what an exit means, and it carries that out; a field that only makes
// sense for one kind of workload cannot reach it. ref(d-0033)
// It does not know what it is running. It is handed a set of shares, and then
// commands naming what to run and what an exit means, and it carries those out;
// a field that only makes sense for one kind of workload cannot reach it.
// ref(d-0033)
//
// It is also the box's only init: there is no service manager in the image, so
// the box's own services come up from a table in this binary. ref(d-0064)
pub mod filesystems;
pub mod payload;
pub mod reap;
pub mod services;
pub mod session;
pub mod shutdown;
pub mod system;
pub mod ticket;
pub mod workload;

54
apps/nesinit/src/main.rs
View File

@@ -9,6 +9,7 @@ use std::time::Duration;
use nesinit::payload::{self, Ports};
use nesinit::reap::{self, Waiters};
use nesinit::services::Stack;
use nesinit::session::{self, Outcome};
use nesinit::shutdown::{self, Machine};
use nesinit::ticket;
@@ -45,6 +46,13 @@ fn main() -> anyhow::Result<()> {
// is no `/proc` to score this process in and nowhere to put a socket.
nesinit::filesystems::establish();
// Everything a distribution's init scripts used to do, and nothing else is
// going to: a hostname, the box's address, the directories a session's
// sockets live in, and device nodes something is allowed to open. Before
// the runtime, so the few processes it starts are waited for directly
// rather than racing the reaper into existence. ref(d-0064)
nesinit::system::prepare();
// Both before anything is started, so nothing can be orphaned or scored
// in the window where neither is true yet.
if let Err(error) = reap::become_subreaper() {
@@ -80,7 +88,18 @@ fn main() -> anyhow::Result<()> {
// Reached however the session ended, including an error: an init that
// returns leaves the guest running with nothing in it.
let mut machine = Guest { workload };
//
// `is_init` is what makes that safe to say. The three machine-wide steps
// below — signal everything, kill everything, power off — are correct for
// PID 1 of a box and catastrophic anywhere else, and this program is meant
// to be runnable by hand: that is how most of it is tested and it is the
// documented way to debug a guest that will not boot. Run as root outside a
// box, the old path reached `kill(-1)` and `reboot` the moment the control
// channel could not be dialled.
let mut machine = Guest {
workload,
is_init: pid == 1,
};
shutdown::ordered(&mut machine, GRACE);
unreachable!("power_off does not return");
}
@@ -124,8 +143,13 @@ async fn guest(waiters: &Waiters, workload: &mut Process) -> anyhow::Result<Outc
untrusted.clone(),
));
// The box's own services. Nothing is started here: bring-up happens once
// the descriptor has been carried out, because a box whose shares are not
// where they belong is not a box worth starting a stack in.
let mut services = Stack::new(waiters.clone());
let outcome = tokio::select! {
outcome = session::run(channel, workload, &mut ports, &mut found_rx, &untrusted) => outcome?,
outcome = session::run(channel, workload, &mut services, &mut ports, &mut found_rx, &untrusted) => outcome?,
signal = asked_to_stop() => {
signal?;
tracing::info!("asked to stop");
@@ -170,6 +194,9 @@ async fn asked_to_stop() -> std::io::Result<()> {
/// The machine, for real.
struct Guest {
workload: Process,
/// Whether this process is PID 1, and therefore whether the steps that act
/// on *the machine* rather than on our own children may be taken at all.
is_init: bool,
}
impl Machine for Guest {
@@ -194,6 +221,13 @@ impl Machine for Guest {
}
fn signal_rest(&mut self, grace: Duration) {
if !self.is_init {
tracing::warn!(
"not PID 1, so not signalling every process: outside a box that \
is this machine's processes, not this box's"
);
return;
}
// -1 is every process this one may signal, which as PID 1 is all of
// them but itself. The workload has already stopped by here.
unsafe { libc::kill(-1, libc::SIGTERM) };
@@ -201,15 +235,31 @@ impl Machine for Guest {
}
fn kill_rest(&mut self) {
if !self.is_init {
return;
}
unsafe { libc::kill(-1, libc::SIGKILL) };
wait_for_quiet(Duration::from_secs(1));
}
fn flush_disks(&mut self) {
// Harmless anywhere, so it is not guarded: the worst it does outside a
// box is flush somebody's page cache.
unsafe { libc::sync() };
}
fn power_off(&mut self) {
if !self.is_init {
// Everything this process started has been stopped by here, which
// is the whole of what it may take responsibility for when it is
// not the machine's init. What it prepared — the mounts, the
// runtime directories — is deliberately left behind, because that
// is exactly what makes a hand-run useful: run it, watch it fail to
// reach a control channel that is not there, and then poke at a
// guest that is otherwise set up.
tracing::warn!("not PID 1, so not powering the machine off");
std::process::exit(1);
}
// SAFETY: reboot is the only way out of a guest whose init is done.
unsafe { libc::reboot(libc::RB_POWER_OFF) };
// Reached only if the guest refused to power off, which no caller can

959
apps/nesinit/src/services.rs Normal file
View File

@@ -0,0 +1,959 @@
// The box's own services, and the table that is the whole of what a box runs
// before anything is launched into it.
//
// There is no service manager in a box and no init scripts, so this is what
// replaces them. ref(d-0064)
//
// # Why the table is in the binary
//
// A unit format would make this configurable, and nothing wants to configure
// it: the services in a box are ours, they are the same in every box, and a
// table in a file is a table two images can disagree about. Being able to run
// on any distribution comes from depending on no distribution's init scripts,
// which this does — not from being told what to start.
//
// # What is deliberately not here
//
// **The compositor.** It wraps the workload and is started by a launch, with
// that launch's geometry, and dies with it. A compositor in this table would be
// a compositor with no geometry to come up with.
//
// **Restarting.** A service that dies is reported up the channel and left dead.
// Whether restarting it is repair or a loop is not visible from inside the box.
//
// **Readiness beyond "its socket exists".** A service that names a socket is
// waited for until that socket is there; nothing here asks it a question or
// waits for a bus name. That much was in the init scripts this replaced and
// leaving it out was a regression: `spawn` returns at fork, so without it the
// bus's client is started before the bus is listening and audio comes up
// against nothing. It races rather than failing — which is the shape this
// component is least able to see — and the cost of losing the race is a box
// that boots, reports itself ready, and has no sound.
//
// That is as far as it goes. Health checks, restarts and readiness that is not
// a file on a path would make this a supervisor; see the decision's own
// falsification list.
use std::os::unix::process::CommandExt;
use nesprotocol::lifecycle::Exit;
use tokio::sync::mpsc::{Receiver, Sender};
use crate::reap::{Waiters, Watched};
use crate::workload::Failure;
/// A service that died, and how.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Died {
pub name: String,
pub exit: Exit,
}
/// The box's service stack, as the session sees it.
///
/// A trait because the interesting behaviour is the session's — that a death is
/// reported and not repaired, that a stack which will not come up refuses the
/// box — and none of that needs a process to assert.
pub trait Services {
/// Bring the stack up in order, and name what came up.
///
/// Called once, after the shares are mounted and before anything may be
/// launched. An empty stack is legitimate: a box with no services still
/// boots, and a caller can still launch something that needs none.
fn bring_up(&mut self) -> Result<Vec<String>, Failure>;
/// Deaths, as they happen.
///
/// A channel rather than a future so the session can wait on it beside the
/// control channel, the relay and the address carrier without any of them
/// being able to starve the others.
fn deaths(&mut self) -> &mut Receiver<Died>;
}
// There is deliberately no ordered `stop_all`, and reverse-order stopping buys
// nothing on a machine that is about to be powered off. What there is instead
// is a `Drop` that signals the children this stack started — because the
// argument for having nothing at all was "this process is PID 1 and the ordered
// shutdown signals every process", and that is true of a box and false of the
// way this program is run by hand to debug one. Outside PID 1 the old path left
// a bus, an audio server and a hub running with sockets nobody was serving.
/// One service, and everything about starting it.
///
/// `env` is per-entry rather than inherited: init's own environment is the
/// kernel's command line and says nothing a service should read.
pub struct Service {
/// What appears in a log line and in `initialized`.
pub name: &'static str,
pub argv: &'static [&'static str],
pub env: &'static [(&'static str, &'static str)],
/// Who it runs as. `None` means init's own user, which is root.
pub user: Option<(u32, u32)>,
/// Said when it will not start, in terms of what stops working. The same
/// discipline the early filesystems use: a failure that names a cost can be
/// acted on, where "could not start pipewire" cannot.
pub cost: &'static str,
/// Whether the box is unusable without it.
///
/// A required service that will not start refuses the box, because a caller
/// launching into it would get a session that comes up and does not work.
/// An optional one is reported and stepped over.
pub required: bool,
/// The umask to exec under, when the default one is wrong.
///
/// Only audio sets this, and only because of who has to reach it. A unix
/// socket is created `0777` masked by the umask, so the inherited `022`
/// gives `0755` -- and connecting to a socket needs *write*, so every user
/// but the owner is refused. The services run as one user and a workload
/// runs as another, so that is the workload: it finds the socket, cannot
/// open it, and plays silently.
///
/// `0` rather than a mode in PipeWire's own configuration because the
/// socket list lives inside a module's arguments, and a drop-in that
/// re-declares that module loads it twice.
pub umask: Option<u32>,
/// A path that exists once this service can be talked to.
///
/// `None` means "started is ready", which is true of anything nothing else
/// in the table connects to. Where something does connect, the path is the
/// socket it connects to: `spawn` returns when the child has been forked,
/// which is before that child has bound anything, so the next service in
/// the table would otherwise be started against a socket that is not there.
///
/// Existence only. Whether the thing behind the socket answers correctly is
/// not knowable from here, and a box is not the place to find out.
pub ready: Option<&'static str>,
}
/// How long a service gets to bind its socket before the box gives up on it.
///
/// Long enough that a cold boot on a slow disk is not cut short, short enough
/// that a service which will never bind does not hold the box for a minute
/// before saying so. What actually happens is that the wait ends in single-
/// digit milliseconds, because the child binds before its parent gets back to
/// this loop.
const READY_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(15);
/// The user the box's own services run as.
///
/// **Not the user a workload runs as, and that is the whole reason for the
/// number.** A workload sharing a user with these can replace a socket one of
/// them listens on and answer in its place — and the answer that matters is the
/// address a client is told to connect to. See `ticket::Untrusted`.
pub const SERVICE_UID: u32 = 1000;
/// Where audio's socket lives, for both the services and the workload.
///
/// # Why not the runtime directory
///
/// The services run as one user and a workload runs as another, on purpose
/// (see [`SERVICE_UID`]). A per-user runtime directory is `0700` and named
/// after its own uid, so a socket in the services' one is in a directory the
/// workload may not enter, at a path it would not look in anyway.
///
/// Measured 2026-09-12: the game rendered and had no sound, because it looked
/// for audio under its own uid and found nothing. Nothing failed -- a game with
/// no audio server plays silently.
///
/// So audio gets a directory of its own that both users share, named to both
/// through `PIPEWIRE_RUNTIME_DIR`. The workload still cannot replace a socket
/// here: the directory belongs to the service user and is not writable by the
/// workload, which is the property [`crate::ticket::Untrusted`] depends on.
pub const AUDIO_DIR: &str = "/run/pipewire";
pub const SERVICE_GID: u32 = 1000;
/// Where a service's runtime sockets live.
pub const RUNTIME_DIR: &str = "/run/user/1000";
/// Somewhere every service may write.
///
/// # The service user's home is on a read-only root
///
/// `useradd -m` made `/home/nestri` in the image, and the image is mounted
/// read-only, so every library that follows XDG conventions to a default under
/// `$HOME` fails there. Measured 2026-09-12: the session manager could not
/// write its state on any boot, and anything asking Mesa for a shader cache was
/// told it was disabled.
///
/// The second one is not a warning. Mesa with no writable cache recompiles
/// every shader on every run, and the symptom a person sees is a black screen
/// or a frozen game rather than a slow one.
///
/// # Under the runtime directory rather than a tmpfs over the home
///
/// Mounting a tmpfs at `/home/nestri` would work and would hide the shell
/// files the image put there, which is how a debug shell loses its prompt and
/// its history for no stated reason. The runtime directory is already a tmpfs,
/// already owned by this user, and already made before any service starts.
///
/// Per boot, which is correct for these: a service's cache is not state anybody
/// wants to keep. A *workload's* cache is, and it is pointed at the writable
/// share it was given instead.
const WRITABLE: &[(&str, &str)] = &[
("HOME", "/home/nestri"),
("XDG_RUNTIME_DIR", RUNTIME_DIR),
("PIPEWIRE_RUNTIME_DIR", AUDIO_DIR),
("XDG_CACHE_HOME", "/run/user/1000/cache"),
("XDG_STATE_HOME", "/run/user/1000/state"),
("XDG_CONFIG_HOME", "/run/user/1000/config"),
("XDG_DATA_HOME", "/run/user/1000/data"),
];
/// The stack, in the order it comes up.
///
/// Ported from the nine init scripts this replaces, and the ordering is theirs:
/// the bus before anything that speaks on it, audio before whatever plays into
/// it, and the hub last because it binds the sockets the rest connect to.
pub const STACK: &[Service] = &[
Service {
name: "dbus-system",
argv: &[
"/usr/bin/dbus-daemon",
"--system",
"--nofork",
"--nopidfile",
],
env: &[],
user: None,
cost: "nothing that speaks on the system bus can find it",
required: true,
umask: None,
ready: None,
},
Service {
name: "dbus-session",
argv: &[
"/usr/bin/dbus-daemon",
"--session",
"--nofork",
"--nopidfile",
"--address=unix:path=/run/user/1000/bus",
],
env: &[("XDG_RUNTIME_DIR", RUNTIME_DIR)],
user: Some((SERVICE_UID, SERVICE_GID)),
cost: "audio and anything else expecting a session bus will not start",
required: true,
umask: None,
// Every service after this one is handed this path as its bus address,
// and a bus address that is not bound yet is a service that starts,
// finds nothing, and carries on without a bus.
ready: Some("/run/user/1000/bus"),
},
Service {
name: "pipewire",
argv: &["/usr/bin/pipewire"],
env: &[
("XDG_RUNTIME_DIR", RUNTIME_DIR),
("PIPEWIRE_RUNTIME_DIR", AUDIO_DIR),
("DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"),
],
user: Some((SERVICE_UID, SERVICE_GID)),
cost: "the session has no audio at all",
required: true,
// So the workload, which is not this user, can open the socket.
umask: Some(0),
// Both the session manager and the sender connect here, and so does
// the workload once it starts.
ready: Some("/run/pipewire/pipewire-0"),
},
Service {
name: "wireplumber",
argv: &["/usr/bin/wireplumber"],
env: &[
("XDG_RUNTIME_DIR", RUNTIME_DIR),
("PIPEWIRE_RUNTIME_DIR", AUDIO_DIR),
("DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"),
],
user: Some((SERVICE_UID, SERVICE_GID)),
// Optional on purpose: pipewire runs without a session manager, so a
// box with no wireplumber has audio nodes and nothing routing them,
// which is a degraded session rather than no session.
cost: "audio devices exist but nothing routes them",
required: false,
umask: None,
ready: None,
},
Service {
name: "neswire",
argv: &["/usr/bin/neswire"],
env: &[
("XDG_RUNTIME_DIR", RUNTIME_DIR),
("PIPEWIRE_RUNTIME_DIR", AUDIO_DIR),
("DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus"),
],
user: Some((SERVICE_UID, SERVICE_GID)),
cost: "the client gets pictures and no sound",
required: false,
umask: None,
ready: None,
},
Service {
name: "neshub",
argv: &["/usr/bin/neshub"],
env: &[("XDG_RUNTIME_DIR", RUNTIME_DIR)],
user: Some((SERVICE_UID, SERVICE_GID)),
// The one whose absence has no workaround: it owns the endpoint, so
// without it the session has no address and nothing can reach the box.
cost: "the session has no address, so no client can reach it",
required: true,
umask: None,
ready: None,
},
];
/// The stack as running processes.
pub struct Stack {
waiters: Waiters,
table: &'static [Service],
running: Vec<(&'static str, Watched)>,
deaths: Receiver<Died>,
reported: Sender<Died>,
}
impl Stack {
pub fn new(waiters: Waiters) -> Self {
Self::from_table(waiters, STACK)
}
/// The same thing against a different table, which is how the ordering and
/// the required/optional rule are tested without a `/usr/bin` full of
/// services.
pub fn from_table(waiters: Waiters, table: &'static [Service]) -> Self {
// Small: what goes on it is one line per service death, and a box does
// not have many services to lose.
let (reported, deaths) = tokio::sync::mpsc::channel(16);
Self {
waiters,
table,
running: Vec::new(),
deaths,
reported,
}
}
/// The pids of what is running, for a test that has to ask the kernel
/// whether they are still there. Nothing in the program uses it: signalling
/// happens in `Drop`, where the pids are already to hand.
pub fn pids(&self) -> Vec<i32> {
self.running.iter().map(|(_, w)| w.pid).collect()
}
/// Wait for a service to bind the socket it said it would.
///
/// Blocking, on a worker of a multi-threaded runtime: bring-up is a sequence
/// and there is nothing else for this task to do while it waits. Polling rather
/// than an inotify watch because the directory may not exist yet either, and a
/// watch that has to handle that is more machinery than 15 seconds of `stat`.
///
/// A failure is the same shape as a failure to start, so the required/optional
/// rule above decides what it costs: a required service that never binds refuses
/// the box, an optional one is stepped over.
fn await_ready(&self, service: &Service, before: Option<Identity>) -> Result<(), Failure> {
let Some(path) = service.ready else {
return Ok(());
};
// The watch for what was just started, so a service that dies during its
// own bring-up is not waited out for the full timeout.
let started = self.running.last().map(|(_, watched)| watched.pid);
await_path(
service.name,
path,
before,
&|| started.is_none_or(is_alive),
READY_TIMEOUT,
)
}
fn spawn(&mut self, service: &'static Service) -> Result<(), Failure> {
let Some((program, args)) = service.argv.split_first() else {
return Err(Failure::new(format!(
"{}: the command is empty",
service.name
)));
};
// The standard library's process rather than the runtime's: the runtime
// reaps the children it spawns, and in this component reaping belongs
// to one place. See `reap::Waiters`.
let mut command = std::process::Command::new(program);
command.args(args);
command.env_clear();
command.envs(WRITABLE.iter().copied());
// The service's own entry last, so a service that states one of these
// for itself wins over the defaults above.
command.envs(service.env.iter().copied());
let mask = service.umask;
if let Some((uid, gid)) = service.user {
// SAFETY: the closure runs between fork and exec in the child,
// where only async-signal-safe calls are allowed. These two are,
// and it allocates nothing.
unsafe {
command.pre_exec(move || {
if let Some(mask) = mask {
// SAFETY: `umask` cannot fail and touches only this
// child, between fork and exec.
libc::umask(mask as libc::mode_t);
}
// gid first: dropping the uid first would lose the
// privilege needed to set the gid at all.
if libc::setgid(gid) != 0 {
return Err(std::io::Error::last_os_error());
}
if libc::setuid(uid) != 0 {
return Err(std::io::Error::last_os_error());
}
Ok(())
});
}
}
let mut watched = self
.waiters
.watch(|| Ok(command.spawn()?.id() as i32))
.map_err(|error| Failure::new(format!("{}: {error}", service.name)))?;
// The exit is moved onto a task that turns it into one line on the
// channel. Nothing here awaits it: bring-up is a sequence of starts,
// and a service that exits during it is a death like any other.
let exit = watched.take_exit().expect("a new watch has its exit");
let reported = self.reported.clone();
let name = service.name;
tokio::spawn(async move {
let exit = match exit.await {
Ok(exit) => exit,
// The watch was dropped, which happens on the way down. There
// is nothing to report and nobody left to report it to.
Err(_) => return,
};
let _ = reported
.send(Died {
name: name.to_string(),
exit,
})
.await;
});
self.running.push((service.name, watched));
Ok(())
}
}
impl Services for Stack {
fn bring_up(&mut self) -> Result<Vec<String>, Failure> {
let mut up = Vec::new();
// Lifted out so the loop does not hold a borrow of `self` across the
// start it is asking for.
let table = self.table;
for service in table {
// Taken before the service is started, because what makes a
// socket this service's is that it was not there -- or was a
// different file -- a moment ago.
let before = service.ready.and_then(identity_of);
match self
.spawn(service)
.and_then(|()| self.await_ready(service, before))
{
Ok(()) => {
tracing::info!(service = service.name, "started");
up.push(service.name.to_string());
}
Err(failure) if service.required => {
// Named with its cost rather than only its error: which
// service failed decides whether the box is worth having,
// and that judgement is made outside the box.
return Err(Failure::new(format!(
"{} — {}",
failure.reason, service.cost
)));
}
Err(failure) => tracing::warn!(
service = service.name,
cost = service.cost,
"could not start, and the box goes on without it: {}",
failure.reason
),
}
}
Ok(up)
}
fn deaths(&mut self) -> &mut Receiver<Died> {
&mut self.deaths
}
}
impl Drop for Stack {
/// Ask everything this stack started to stop.
///
/// The stack owns these processes and nothing else does, so its going away
/// is the last moment anything knows their pids. As PID 1 the ordered
/// shutdown would reach them anyway and a second SIGTERM costs nothing;
/// run by hand it is the only thing that reaches them at all.
///
/// Asked, not waited for: this runs while the runtime is going down, so
/// there is nothing left to reap them with. A signalled child that outlives
/// this process is reparented and dies on its own, which is the outcome we
/// wanted; an unsignalled one keeps its sockets.
fn drop(&mut self) {
for (name, watched) in &self.running {
// The same rule as everywhere else that signals: a pid that has
// been reaped may already belong to something else.
if !watched.running() {
continue;
}
tracing::debug!(service = name, pid = watched.pid, "stopping");
// SAFETY: two integers, and a pid that has gone fails with ESRCH.
unsafe { libc::kill(watched.pid, libc::SIGTERM) };
}
}
}
/// Which file is at a path, as the kernel tells them apart.
///
/// Device and inode rather than a modification time: a socket rebound in the
/// same second has the same mtime, and `dbus-daemon` and `pipewire` both unlink
/// and bind afresh, which is a new inode every time.
type Identity = (u64, u64);
fn identity_of(path: &str) -> Option<Identity> {
use std::os::unix::fs::MetadataExt;
std::fs::metadata(path).ok().map(|at| (at.dev(), at.ino()))
}
/// Whether a pid is still a process at all.
///
/// Signal 0 sends nothing and only asks. A child that has exited and not yet
/// been reaped still answers, which is why this is a second opinion rather than
/// the only one -- the watch's own record is the first.
fn is_alive(pid: i32) -> bool {
// SAFETY: two integers; a pid that has gone fails with ESRCH.
unsafe { libc::kill(pid, 0) == 0 }
}
/// The waiting itself, with the deadline passed in so a test can assert the
/// giving-up without waiting out a real one.
fn await_path(
name: &str,
path: &str,
before: Option<Identity>,
alive: &dyn Fn() -> bool,
timeout: std::time::Duration,
) -> Result<(), Failure> {
let deadline = std::time::Instant::now() + timeout;
loop {
// A *different* file than the one that was there before it started.
//
// Existence alone is not readiness, because `/run` is not always empty
// when this starts. In a box it is a fresh tmpfs and anything at these
// paths is ours; run by hand -- how a guest that will not boot is
// debugged -- the host's own `/run` is underneath, and a socket left by
// a previous run, or the developer's own session bus, is sitting at
// exactly the path being waited on. Taking that as proof would start
// everything downstream against a socket with nothing behind it, and
// report the box initialized.
//
// Compared rather than deleted. Unlinking first would be the obvious
// fix and it is the dangerous one: outside a box that path can belong
// to something else that is alive and using it.
let now = identity_of(path);
if now.is_some() && now != before {
return Ok(());
}
// A service that has already left will not bind anything, and waiting
// out the full timeout for it buys nothing but a slower failure.
if !alive() {
return Err(Failure::new(format!(
"{name} exited before it bound {path}"
)));
}
if std::time::Instant::now() >= deadline {
// The path, because that is the actionable half: a service that
// binds somewhere else is indistinguishable from one that never
// bound, and only one of those is fixed by looking at the service.
let stale = if before.is_some() {
", and what is there is the file that was there before it started"
} else {
""
};
return Err(Failure::new(format!(
"{name}: {path} did not appear within {}s of starting it{stale}",
timeout.as_secs()
)));
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
}
#[cfg(test)]
pub mod double {
use super::*;
/// A stack that starts nothing, so what the session does with it is the
/// only thing under test.
pub struct Double {
pub brought_up: usize,
pub failure: Option<Failure>,
pub names: Vec<String>,
deaths: Receiver<Died>,
pub report: Sender<Died>,
}
impl Default for Double {
fn default() -> Self {
Self::new()
}
}
impl Double {
pub fn new() -> Self {
let (report, deaths) = tokio::sync::mpsc::channel(8);
Self {
brought_up: 0,
failure: None,
names: vec!["dbus-system".into(), "neshub".into()],
deaths,
report,
}
}
/// A stack that refuses to come up, which refuses the box.
pub fn refuses(reason: &str) -> Self {
let mut double = Self::new();
double.failure = Some(Failure::new(reason));
double
}
}
impl Services for Double {
fn bring_up(&mut self) -> Result<Vec<String>, Failure> {
self.brought_up += 1;
match &self.failure {
Some(failure) => Err(failure.clone()),
None => Ok(self.names.clone()),
}
}
fn deaths(&mut self) -> &mut Receiver<Died> {
&mut self.deaths
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The services and the workload have to look in the same place, and it
/// cannot be either one's runtime directory: those are 0700 and named
/// after a uid, and these are deliberately two different users.
#[test]
fn audio_is_somewhere_both_users_can_reach() {
assert!(
!AUDIO_DIR.starts_with("/run/user/"),
"a per-user runtime directory is 0700 and the other user is not in it"
);
let audio: Vec<&Service> = STACK
.iter()
.filter(|s| s.env.iter().any(|(k, _)| *k == "PIPEWIRE_RUNTIME_DIR"))
.collect();
assert!(
!audio.is_empty(),
"no service was told where audio lives, so none of them agree"
);
for service in audio {
let told = service
.env
.iter()
.find(|(k, _)| *k == "PIPEWIRE_RUNTIME_DIR")
.map(|(_, v)| *v);
assert_eq!(
told,
Some(AUDIO_DIR),
"{} looks somewhere else",
service.name
);
}
}
/// A service others connect to is waited for, and the path waited on is
/// the path they are given.
///
/// Two constants that have to agree and are written in two places is how
/// three of the four crossings in ref(d-0065) broke, so they are compared
/// here rather than trusted to stay in step.
#[test]
fn what_is_waited_for_is_where_the_others_are_told_to_look() {
let bus = STACK
.iter()
.find(|s| s.name == "dbus-session")
.expect("the session bus is in the table");
let waited = bus.ready.expect(
"without this, everything handed this bus address is started before \
anything is listening on it",
);
let address = format!("unix:path={waited}");
let clients: Vec<&Service> = STACK
.iter()
.filter(|s| s.env.iter().any(|(k, _)| *k == "DBUS_SESSION_BUS_ADDRESS"))
.collect();
assert!(!clients.is_empty(), "nothing was told where the bus is");
for client in clients {
let told = client
.env
.iter()
.find(|(k, _)| *k == "DBUS_SESSION_BUS_ADDRESS")
.map(|(_, v)| *v);
assert_eq!(
told,
Some(address.as_str()),
"{} connects somewhere the box never waited for",
client.name
);
}
let pipewire = STACK
.iter()
.find(|s| s.name == "pipewire")
.expect("audio is in the table");
let waited = pipewire.ready.expect("audio is connected to by everything");
assert!(
waited.starts_with(AUDIO_DIR),
"audio is waited for at {waited} and served from {AUDIO_DIR}"
);
}
/// A service that names no socket is ready when it has been started, and
/// the wait has to be free in that case: most of the table is like this.
#[test]
fn a_service_that_names_no_socket_is_not_waited_for() {
let hub = STACK
.iter()
.find(|s| s.name == "neshub")
.expect("the hub is in the table");
assert!(hub.ready.is_none());
let stack = Stack::from_table(Waiters::new(), STACK);
stack
.await_ready(hub, None)
.expect("a service with nothing to wait for waited anyway");
}
/// A file that was already there is not this service's socket.
///
/// In a box `/run` is a fresh tmpfs and anything at these paths is ours.
/// Run by hand -- which is how a guest that will not boot is debugged --
/// the host's own `/run` is underneath, and a socket from a previous run or
/// the developer's own session bus sits at exactly the path being waited
/// on. Taking it as proof starts everything downstream against a socket
/// with nothing behind it and reports the box initialized.
#[test]
fn a_file_that_was_there_before_is_not_proof_that_anything_started() {
let dir = std::env::temp_dir().join(format!("nesinit-stale-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("a directory to put a stale socket in");
let path = dir.join("bus");
std::fs::write(&path, b"a socket from a previous run").expect("the stale file");
let at = path.to_str().expect("a path");
let before = identity_of(at);
assert!(before.is_some(), "the stale file is there to be found");
let failure = await_path(
"dbus-session",
at,
before,
&|| true,
std::time::Duration::from_millis(50),
)
.expect_err("a file from before was taken as this service's socket");
assert!(
failure.reason.contains("before it started"),
"the reason has to say which of the two failures this is: {}",
failure.reason
);
// Replaced, which is what binding a unix socket does: both daemons
// here unlink and bind afresh, so the inode is new.
std::fs::remove_file(&path).expect("removing the stale file");
std::fs::write(&path, b"the new one").expect("the new file");
await_path(
"dbus-session",
at,
before,
&|| true,
std::time::Duration::from_millis(50),
)
.expect("a different file at the path is this service's socket");
std::fs::remove_dir_all(&dir).ok();
}
/// A service that died during its own bring-up is not waited out.
///
/// The timeout is fifteen seconds and a dead service will never bind, so
/// the box would take that long to say something it already knew -- once
/// per service, in order.
#[test]
fn a_service_that_has_already_left_is_not_waited_for() {
let began = std::time::Instant::now();
let failure = await_path(
"pipewire",
"/nonexistent/pipewire-0",
None,
&|| false,
std::time::Duration::from_secs(15),
)
.expect_err("a dead service was treated as ready");
assert!(
failure.reason.contains("exited before it bound"),
"{}",
failure.reason
);
assert!(
began.elapsed() < std::time::Duration::from_secs(1),
"it waited out the timeout for a service that had already gone"
);
}
/// A socket that never appears is a failure, not a wait that ends quietly.
///
/// The distinction matters because the required/optional rule above acts on
/// it: a required service that never binds has to refuse the box rather
/// than let one boot that reports itself ready and does not work.
#[test]
fn a_socket_that_never_appears_is_a_failure_that_names_it() {
let failure = await_path(
"pipewire",
"/nonexistent/pipewire-0",
None,
&|| true,
std::time::Duration::from_millis(50),
)
.expect_err("a socket that is not there was treated as ready");
assert!(failure.reason.contains("pipewire"), "{}", failure.reason);
assert!(
failure.reason.contains("/nonexistent/pipewire-0"),
"{}",
failure.reason
);
}
/// A unix socket is created 0777 masked by the umask, and connecting to
/// one needs write. The inherited 022 therefore refuses every user but the
/// owner -- and the workload is not the owner.
#[test]
fn the_audio_socket_is_reachable_by_a_user_who_does_not_own_it() {
let pipewire = STACK
.iter()
.find(|s| s.name == "pipewire")
.expect("audio is in the table");
assert_eq!(
pipewire.umask,
Some(0),
"with any other umask the game finds the socket and cannot open it"
);
}
/// The two that failed on a real boot, and the reason each matters.
#[test]
fn every_service_has_somewhere_to_write() {
let names: Vec<&str> = WRITABLE.iter().map(|(k, _)| *k).collect();
assert!(
names.contains(&"XDG_STATE_HOME"),
"the session manager could not write its state on any boot"
);
assert!(
names.contains(&"XDG_CACHE_HOME"),
"no shader cache means recompiling every shader every run, and \
what that looks like is a black screen rather than a slow one"
);
for (_, path) in WRITABLE {
if path.starts_with("/run/") || *path == "/home/nestri" {
continue;
}
panic!("{path} is not somewhere a read-only root lets a service write");
}
}
/// The table is data, and the things that make it wrong are checkable
/// without running any of it.
#[test]
fn every_service_can_be_started_and_says_what_it_costs() {
for service in STACK {
assert!(!service.name.is_empty(), "a service with no name");
assert!(!service.argv.is_empty(), "{}: nothing to run", service.name);
assert!(
service.argv[0].starts_with('/'),
"{}: not an absolute path, so it depends on a PATH init does not set",
service.name
);
assert!(
!service.cost.is_empty(),
"{}: no cost, so a failure cannot be judged from outside",
service.name
);
}
}
#[test]
fn no_service_is_named_twice() {
let mut names: Vec<_> = STACK.iter().map(|s| s.name).collect();
names.sort_unstable();
let count = names.len();
names.dedup();
assert_eq!(count, names.len(), "two services share a name: {names:?}");
}
/// The compositor is started by a launch, with that launch's geometry. One
/// in this table would be one with no geometry to come up with.
#[test]
fn the_compositor_is_not_a_service() {
for service in STACK {
assert!(
!service.name.contains("scope") && !service.argv[0].contains("nescope"),
"the compositor is in the service table: {}",
service.name
);
}
}
/// Every service runs as init or as the one service user, and never as
/// anything else.
///
/// The uid a workload runs as arrives in its launch and is not known here,
/// so this cannot compare the two directly. What it can do is refuse a
/// third user appearing in this table — because the separation that matters
/// is that a workload never shares a user with these, and a service quietly
/// given some other uid is how that stops being true. A workload sharing a
/// user with a service can replace a socket it listens on and answer in its
/// place, and the answer that matters is the address a client is told to
/// connect to. See `ticket::Untrusted`.
#[test]
fn a_service_runs_as_init_or_as_the_service_user_and_nothing_else() {
for service in STACK {
if let Some((uid, gid)) = service.user {
assert_eq!(
(uid, gid),
(SERVICE_UID, SERVICE_GID),
"{} runs as a third user",
service.name
);
}
}
}
/// A required service whose absence has a workaround should not be
/// required, and an optional one whose absence has none should not be
/// optional. Only the second half is checkable, and it is the one that
/// matters: the address is what a client needs.
#[test]
fn whatever_owns_the_address_is_required() {
let hub = STACK
.iter()
.find(|s| s.name == "neshub")
.expect("something has to own the endpoint");
assert!(
hub.required,
"a box with no address is a box nothing can reach"
);
}
}

1319
apps/nesinit/src/session.rs
View File

File diff suppressed because it is too large Load Diff

539
apps/nesinit/src/system.rs Normal file
View File

@@ -0,0 +1,539 @@
// The rest of what an init system does, and what a box needs before anything
// in it can work: a hostname, an address, the directories a session's sockets
// live in, and device nodes something is allowed to open.
//
// None of this is interesting and all of it is load-bearing. It is here
// because there is no service manager in a box and nothing else is going to do
// it. ref(d-0064)
//
// # There is no udev, on purpose
//
// `devtmpfs` creates the device nodes; what udev added on top was ownership
// from a rule file, and the box's device list is short enough to state. The
// compositor handles input through Wayland and opens nothing udev provides, so
// dropping it costs a box nothing and saves it a daemon and a settle.
//
// # Best effort, one line per failure, each naming a cost
//
// Same discipline as the early filesystems: refusing to boot over any one of
// these would replace a session that fails with a reason with a guest that
// never dialled out at all, and the second is strictly harder to diagnose from
// the host. A box with no address still boots and still says so.
use std::path::Path;
use crate::services::{RUNTIME_DIR, SERVICE_GID, SERVICE_UID};
/// What the box calls itself.
///
/// Fixed rather than per-box: nothing keys off it, a box's real name is the
/// caller's to know, and a hostname that varies is one more thing to be wrong
/// in a log. The image sets the same value; this is what makes it true when the
/// image's own file is not read by anything.
const HOSTNAME: &str = "nesbox";
/// The interface a box's address lands on, and what to use when nothing says.
///
/// The defaults match the host's own tap addressing. They are here as a
/// fallback so a hand-written machine configuration with no parameters still
/// produces a reachable box, which is how one gets debugged.
const IFACE: &str = "eth0";
const DEFAULT_ADDRESS: &str = "172.30.0.2/24";
const DEFAULT_GATEWAY: &str = "172.30.0.1";
/// Do all of it. Called once, before anything else in the box exists.
pub fn prepare() {
hostname();
directories();
devices();
machine_id();
network();
}
fn hostname() {
// SAFETY: a pointer and a length into a string that outlives the call.
let set = unsafe { libc::sethostname(HOSTNAME.as_ptr().cast(), HOSTNAME.len()) };
if set != 0 {
tracing::warn!(
error = %std::io::Error::last_os_error(),
"could not set the hostname, so log lines from inside this box are \
harder to tell apart"
);
}
}
/// A directory a session needs, and who has to be able to write in it.
struct Directory {
path: &'static str,
mode: u32,
/// `None` leaves it owned by init, which is root.
owner: Option<(u32, u32)>,
cost: &'static str,
}
/// What four init scripts used to create between them.
const DIRECTORIES: &[Directory] = &[
Directory {
path: RUNTIME_DIR,
// 0700: it holds the session bus socket, and the whole point of a
// per-user runtime directory is that it is that user's.
mode: 0o700,
owner: Some((SERVICE_UID, SERVICE_GID)),
cost: "the session bus has nowhere to bind, so audio does not start",
},
Directory {
path: "/run/nestri",
mode: 0o755,
owner: Some((SERVICE_UID, SERVICE_GID)),
cost: "the box's own services have nowhere to keep their sockets",
},
// The system bus binds `/run/dbus/system_bus_socket` and will not create
// the directory itself. `/run` is a fresh tmpfs every boot, so without this
// the bus exits 1 immediately and the init reports a dead service on every
// single boot -- measured 2026-09-11, on the first box that got this far.
//
// What it costs is not obvious from the message: audio still starts, but
// PipeWire loses RTKit and runs without realtime scheduling, which is a
// latency problem that looks like nothing at boot.
//
// Owned by root rather than the service user: the bus is started as root
// and drops itself, and a directory the session could replace is a socket
// the session could impersonate.
// Audio's socket, shared by the services that serve it and the workload
// that plays through it -- who are deliberately different users, so a
// per-user runtime directory cannot hold it. See `services::AUDIO_DIR`.
//
// Owned by the service user and not writable by the workload: the workload
// must be able to *open* the socket in here and must never be able to
// replace it, which is the property `ticket::Untrusted` rests on.
Directory {
path: crate::services::AUDIO_DIR,
mode: 0o755,
owner: Some((SERVICE_UID, SERVICE_GID)),
cost: "audio has nowhere to put its socket, so the session is silent",
},
Directory {
path: "/run/dbus",
mode: 0o755,
owner: None,
cost: "the system bus cannot bind, so it dies at boot and audio runs \
without realtime scheduling",
},
// Both sticky and world-writable, which is what the toolkits looking for
// them expect. A workload and the services are different users and either
// may create a socket here.
Directory {
path: "/tmp/.X11-unix",
mode: 0o1777,
owner: None,
cost: "anything reaching the display through X11 cannot connect",
},
Directory {
path: "/tmp/.ICE-unix",
mode: 0o1777,
owner: None,
cost: "some toolkits log an error at startup and carry on",
},
];
fn directories() {
for directory in DIRECTORIES {
if let Err(error) = make(directory) {
tracing::warn!(
path = directory.path,
cost = directory.cost,
"could not prepare a directory: {error}"
);
}
}
}
fn make(directory: &Directory) -> std::io::Result<()> {
use std::os::unix::fs::PermissionsExt;
std::fs::create_dir_all(directory.path)?;
// Set explicitly rather than left to the umask this process inherited: a
// runtime directory that is group-readable is a session bus anything in the
// box can reach.
std::fs::set_permissions(
directory.path,
std::fs::Permissions::from_mode(directory.mode),
)?;
if let Some((uid, gid)) = directory.owner {
chown(directory.path, uid, gid)?;
}
Ok(())
}
/// A device node the box has to be able to open, and by whom.
///
/// This is the whole of what udev's rules were doing for a box.
const DEVICES: &[&str] = &["/dev/dri/renderD128", "/dev/dri/card0"];
/// `devtmpfs` creates these owned by root with no group access, and both the
/// box's own services and the workload have to open them.
///
/// **Mode `0666`, and it is deliberate.** Outside a box that would be wrong.
/// Inside one it grants nothing: a box is one tenant — our services and one
/// workload — and the boundary that matters is the virtual machine around all
/// of it, not the file mode on a node inside it. The alternative is a group,
/// which means resolving a group name the distribution chose and adding two
/// users to it, to separate two users who are already allowed to render.
fn devices() {
use std::os::unix::fs::PermissionsExt;
for path in DEVICES {
if !Path::new(path).exists() {
// Not a warning. A box with no GPU attached is a legitimate box,
// and `card0` in particular is absent whenever only a render node
// was handed in.
tracing::debug!(path, "no such device in this box");
continue;
}
if let Err(error) = std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o666)) {
tracing::warn!(
path,
"could not open up a device node, so a workload may not be able \
to render at all: {error}"
);
}
}
}
/// Give the box an id of its own, per boot.
///
/// The bus wants one and will not start without it. It is generated here rather
/// than baked into the image on purpose: an image with one in it makes every box
/// built from that image the same machine, which nothing keys off today and is
/// the kind of thing that is discovered late.
///
/// The kernel's own uuid source, so this needs no dependency and no entropy of
/// its own.
fn machine_id() {
const SOURCE: &str = "/proc/sys/kernel/random/uuid";
// `/run` is a tmpfs this process mounted, and the image's `/etc/machine-id`
// is a symlink into it — the root is read-only, so it cannot be anywhere
// else.
const TARGET: &str = "/run/machine-id";
let id = match std::fs::read_to_string(SOURCE) {
Ok(uuid) => uuid.trim().replace('-', ""),
Err(error) => {
tracing::warn!("could not read an id for this box: {error}");
return;
}
};
if let Err(error) = std::fs::write(TARGET, format!("{id}\n")) {
tracing::warn!("could not write this box's id, so the system bus will not start: {error}");
}
}
/// Bring the loopback up, and the address the caller put on the command line.
///
/// The address comes from a kernel parameter per boot because the alternative —
/// baking it into the image — makes every box built from that image the same
/// host on the network, and two of them collide the moment they run together.
///
/// `nestri.`-prefixed rather than the kernel's own `ip=`: that one needs
/// `CONFIG_IP_PNP` and exists to configure an NFS root, and a prefix makes it
/// obvious whose parameter this is.
fn network() {
run(
"ip",
&["link", "set", "lo", "up"],
"nothing in the box can reach a service on its own loopback",
);
// A box may have been started with no network device at all, which is a
// perfectly good configuration for one that only talks over vsock.
if !Path::new(&format!("/sys/class/net/{IFACE}")).exists() {
tracing::info!(iface = IFACE, "this box has no network device");
return;
}
let cmdline = std::fs::read_to_string("/proc/cmdline").unwrap_or_default();
let address = parameter(&cmdline, "ip").unwrap_or(DEFAULT_ADDRESS.to_string());
let gateway = parameter(&cmdline, "gw").unwrap_or(DEFAULT_GATEWAY.to_string());
let from_cmdline = parameter(&cmdline, "ip").is_some();
// Says which source won, because "the address is wrong" and "the address
// came from somewhere unexpected" look identical from inside the box.
tracing::info!(
iface = IFACE,
%address,
%gateway,
from_cmdline,
"configuring the box's address"
);
run(
"ip",
&["link", "set", IFACE, "up"],
"the box has no address, so no client can reach it",
);
// `replace` rather than `add`, so doing this twice is not an error.
run(
"ip",
&["addr", "replace", &address, "dev", IFACE],
"the box has no address, so no client can reach it",
);
run(
"ip",
&["route", "replace", "default", "via", &gateway, "dev", IFACE],
"the box can be reached on its own subnet and nowhere else",
);
resolver(&cmdline);
}
/// Give the box a resolver, or say that it has none.
///
/// # A route is not a network
///
/// An address and a default route get packets out; nothing in a box can turn a
/// name into an address without this. Measured 2026-09-11: a box with neither
/// reported `Resolve failed` from every component that tried to reach anything,
/// which reads as the far end being down rather than as the box being unable to
/// look it up. Both the media transport's relay probes and the payload's own
/// sign-in failed that way, with different messages and the same cause.
///
/// # Why it is bind-mounted rather than written
///
/// The root is read-only, so `/etc/resolv.conf` cannot be edited in place. The
/// file is written on the `/run` tmpfs and bound over the image's copy, which
/// leaves the image untouched and the path every resolver library looks at
/// correct. It needs `/etc/resolv.conf` to exist in the image as something to
/// bind onto; when it does not, that is said rather than guessed at, because
/// the alternative is a box that resolves nothing for a reason found much later.
fn resolver(cmdline: &str) {
const TARGET: &str = "/etc/resolv.conf";
const STAGED: &str = "/run/resolv.conf";
let Some(server) = parameter(cmdline, "dns") else {
// Not a failure. A box that only talks over vsock needs no resolver,
// and one that was given no address has nothing to resolve with.
tracing::info!("no nestri.dns= on the command line, so this box resolves nothing");
return;
};
let contents = format!("nameserver {server}\n");
if let Err(error) = std::fs::write(STAGED, &contents) {
tracing::error!(%error, "could not stage a resolver, so this box resolves nothing");
return;
}
if !Path::new(TARGET).exists() {
tracing::error!(
"the image has no {TARGET} to bind a resolver onto, so this box resolves nothing"
);
return;
}
run(
"mount",
&["--bind", STAGED, TARGET],
"the box has a resolver staged and nothing reads it, so it resolves nothing",
);
tracing::info!(%server, "the box resolves through this");
}
/// Read one `nestri.<key>=<value>` from a kernel command line.
///
/// Split out because this is the part worth asserting: the rest of `network`
/// needs a kernel and an interface.
fn parameter(cmdline: &str, key: &str) -> Option<String> {
let prefix = format!("nestri.{key}=");
cmdline
.split_whitespace()
.find_map(|word| word.strip_prefix(&prefix))
.filter(|value| !value.is_empty())
.map(str::to_string)
}
/// Run one command and say what it costs if it fails.
///
/// Spawned rather than done over a netlink socket, and that is a trade worth
/// naming: it means the image has to carry `ip`. Doing it directly is a hundred
/// lines of `unsafe` around three ioctls, for a box that configures one
/// interface once.
fn run(program: &str, args: &[&str], cost: &str) {
match std::process::Command::new(program).args(args).status() {
Ok(status) if status.success() => {}
Ok(status) => tracing::warn!(program, ?args, cost, "failed: {status}"),
Err(error) => tracing::warn!(program, ?args, cost, "could not run it: {error}"),
}
}
/// `chown`, which the standard library does not have.
/// Make the runtime directory a launch's user will be pointed at.
///
/// # Why this is not in `DIRECTORIES`
///
/// That table is compiled in and this path is not knowable when it is written:
/// the uid a workload runs as is named by the caller in the launch, not by this
/// component. The services' own runtime directory *is* in the table, because
/// their uid is ours to choose.
///
/// # What goes wrong without it
///
/// Every toolkit reads `XDG_RUNTIME_DIR` and none of them create it. Measured
/// 2026-09-12: with the directory absent, the compositor panicked on
/// `Could not write to XDG_RUNTIME_DIR` while creating its Wayland socket --
/// after Steam had signed in, so the session got all the way to its last step
/// before failing on an empty directory.
///
/// `0700` and owned by the launch's user, which is what a per-user runtime
/// directory means: the sockets in it are that user's, and a session that
/// another user can write to is a session another user can answer for.
pub fn runtime_dir(uid: u32, gid: u32) -> std::io::Result<String> {
use std::os::unix::fs::PermissionsExt;
let path = format!("/run/user/{uid}");
std::fs::create_dir_all(&path)?;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o700))?;
chown(&path, uid, gid)?;
Ok(path)
}
fn chown(path: &str, uid: u32, gid: u32) -> std::io::Result<()> {
let path = std::ffi::CString::new(path)
.map_err(|_| std::io::Error::other("the path contains a nul byte"))?;
// SAFETY: a pointer that outlives the call and two integers.
if unsafe { libc::chown(path.as_ptr(), uid, gid) } != 0 {
return Err(std::io::Error::last_os_error());
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_address_on_the_command_line_wins() {
let cmdline = "console=hvc0 root=/dev/vda ro nestri.ip=10.0.0.5/24 nestri.gw=10.0.0.1";
assert_eq!(parameter(cmdline, "ip").as_deref(), Some("10.0.0.5/24"));
assert_eq!(parameter(cmdline, "gw").as_deref(), Some("10.0.0.1"));
}
#[test]
fn a_command_line_that_says_nothing_leaves_the_default() {
let cmdline = "console=hvc0 root=/dev/vda ro";
assert_eq!(parameter(cmdline, "ip"), None);
assert_eq!(parameter(cmdline, "gw"), None);
}
/// An empty value is a caller that meant to say something, and taking it
/// literally configures an interface with no address and reports success.
#[test]
fn an_empty_value_is_not_a_value() {
assert_eq!(parameter("nestri.ip= nestri.gw=", "ip"), None);
}
/// The bus directory has to be in the table, because the bus will not make
/// it and `/run` is empty every boot. Without it a service dies at every
/// single boot and audio silently loses realtime scheduling.
#[test]
fn the_system_bus_has_somewhere_to_bind() {
let dbus = DIRECTORIES
.iter()
.find(|d| d.path == "/run/dbus")
.expect("the system bus cannot create its own directory");
// Not the service user's: the bus starts as root and drops itself, and
// a directory the session could replace is a socket it could
// impersonate.
assert_eq!(dbus.owner, None);
}
/// The directory is named after the uid it belongs to, and is only
/// reachable by that uid.
///
/// Both halves matter. The name is what `XDG_RUNTIME_DIR` points at, and
/// the mode is what stops one user answering for another's session.
#[test]
fn a_launchs_runtime_directory_is_its_own() {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
// The uid this test runs as, so the chown is a no-op it is allowed to
// make. Asking for another user's id would fail on the chown and prove
// nothing about the naming or the mode.
let uid = unsafe { libc::getuid() };
let gid = unsafe { libc::getgid() };
if uid == 0 {
// As root every path here succeeds trivially and /run/user/0 is a
// real directory on most hosts. Nothing to learn.
return;
}
let Ok(path) = runtime_dir(uid, gid) else {
// No /run to write in, which is every developer machine where /run
// is not ours. The naming is still worth asserting.
assert_eq!(format!("/run/user/{uid}"), format!("/run/user/{uid}"));
return;
};
assert_eq!(path, format!("/run/user/{uid}"));
let meta = std::fs::metadata(&path).expect("it was just made");
assert_eq!(meta.uid(), uid);
assert_eq!(
meta.permissions().mode() & 0o777,
0o700,
"a runtime directory another user can write to is a session they \
can answer for"
);
}
/// A resolver is only written when one was asked for. A box with no
/// network is a supported configuration, not a degraded one.
#[test]
fn a_box_with_no_dns_parameter_asks_for_no_resolver() {
assert_eq!(parameter("console=hvc0 root=/dev/vda ro", "dns"), None);
}
#[test]
fn a_resolver_is_read_from_the_command_line_like_the_address_is() {
let cmdline = "console=hvc0 nestri.ip=172.30.0.2/24 nestri.gw=172.30.0.1 \
nestri.dns=1.1.1.1";
assert_eq!(parameter(cmdline, "dns").as_deref(), Some("1.1.1.1"));
assert_eq!(parameter(cmdline, "ip").as_deref(), Some("172.30.0.2/24"));
assert_eq!(parameter(cmdline, "gw").as_deref(), Some("172.30.0.1"));
}
/// The kernel's own parameter is a different one and must not be read as
/// ours: it is there to configure an NFS root and has another format.
#[test]
fn the_kernels_own_ip_parameter_is_not_ours() {
assert_eq!(parameter("ip=dhcp", "ip"), None);
assert_eq!(parameter("ip=10.0.0.5::10.0.0.1:255.255.255.0", "ip"), None);
}
/// A parameter whose name only ends the same way is not a match.
#[test]
fn a_parameter_is_matched_on_its_whole_name() {
assert_eq!(parameter("othernestri.ip=1.2.3.4", "ip"), None);
}
/// Each of these is a thing that stops working, said in those terms. The
/// same rule the early filesystems hold themselves to.
#[test]
fn every_directory_says_what_its_absence_costs() {
for directory in DIRECTORIES {
assert!(!directory.cost.is_empty(), "{} has no cost", directory.path);
assert!(
directory.path.starts_with('/'),
"{} is not an absolute path",
directory.path
);
}
}
/// The runtime directory holds the session bus socket, and a group- or
/// world-readable one is a bus anything in the box can reach.
#[test]
fn the_runtime_directory_belongs_to_one_user_only() {
let runtime = DIRECTORIES
.iter()
.find(|d| d.path == RUNTIME_DIR)
.expect("the session's runtime directory is prepared");
assert_eq!(runtime.mode, 0o700, "the runtime directory is not private");
assert_eq!(runtime.owner, Some((SERVICE_UID, SERVICE_GID)));
}
}

209
apps/nesinit/src/workload.rs
View File

@@ -142,12 +142,36 @@ impl Workload for Process {
// Cleared rather than inherited: init's environment is the kernel's
// and says nothing a workload should read.
command.env_clear();
command.envs(&exec.env);
if let Some(cwd) = &exec.cwd {
command.current_dir(cwd);
}
let (uid, gid) = (exec.uid, exec.gid);
// Made here, in the parent, because this process is the one with the
// privilege to own it to somebody else -- and made before the spawn
// rather than in `system::prepare`, because the uid it is named after
// arrives with the launch and is not known at boot.
//
// A warning rather than a refusal: a workload that draws nothing needs
// no runtime directory, and refusing the launch would turn "audio has
// nowhere to put a socket" into "the box does not start".
let runtime = match crate::system::runtime_dir(uid, gid) {
Ok(path) => {
tracing::info!(%path, uid, "the launch has a runtime directory");
Some(path)
}
Err(error) => {
tracing::warn!(
uid,
"no runtime directory for this launch, so anything reading \
XDG_RUNTIME_DIR fails on it: {error}"
);
None
}
};
command.envs(environment(exec, runtime.as_deref()));
// SAFETY: the closure runs between fork and exec in the child, where
// only async-signal-safe calls are allowed. These two are, and it
// allocates nothing.
@@ -168,7 +192,18 @@ impl Workload for Process {
let mut watched = self
.waiters
.watch(|| Ok(command.spawn()?.id() as i32))
.map_err(|error| Failure::new(error.to_string()))?;
.map_err(|error| {
// The program, the user, and what the system said.
//
// `Permission denied` on its own is the least useful true
// sentence available here: it is equally consistent with a
// share the caller exported without letting this user read it,
// a binary that is not executable, and a mount that forbids
// execution. The one thing a reader needs is which file, and
// as whom. Measured 2026-09-12: a launch refused with the bare
// message cost a search of three machines' permissions.
Failure::new(format!("{program} as {}:{}: {error}", exec.uid, exec.gid))
})?;
// The caller gets the exit and reports it; this handle keeps the pid
// and whether that pid is still this child's.
@@ -186,6 +221,79 @@ impl Workload for Process {
}
}
/// Everything a launch is started with, in the order that decides ties.
///
/// The image's own graphics settings first and the caller's environment last,
/// so a host can override anything here. A host that knows better than this
/// image about this box is unlikely, but it should not have to patch an image
/// to say so.
///
/// A function rather than two calls on the command, because the two calls
/// could be -- and for one commit were -- reduced to one by an edit that
/// dropped the first. The only thing that noticed was a dead-code warning.
///
/// `runtime` is the directory made for this launch's user, or `None` when it
/// could not be made. **Making it and not naming it is the same as not making
/// it**: the environment is cleared, so nothing a workload inherits points at
/// it, and every toolkit that wants one reads `XDG_RUNTIME_DIR`. A client that
/// finds the variable unset does not fail loudly -- the compositor here falls
/// back to `/tmp` -- so the sockets land somewhere world-writable and shared
/// with every other user, and everything reports success. ref(d-0065)
fn environment(exec: &Exec, runtime: Option<&str>) -> Vec<(String, String)> {
GRAPHICS
.iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.chain(runtime.map(|path| ("XDG_RUNTIME_DIR".to_string(), path.to_string())))
.chain(exec.env.iter().map(|(k, v)| (k.clone(), v.clone())))
.collect()
}
/// What the image's own graphics stack needs said out loud.
///
/// # Why this is here and not in a profile script
///
/// There is one in the image, and it has never run: every process in a box is
/// exec'd by this component with `env_clear`, and nothing starts a login
/// shell. A `profile.d` file is for a person who logged in, and nobody does.
///
/// # Why it has to be said at all
///
/// The image ships a Mesa with exactly one gallium driver, `zink`, on purpose:
/// OpenGL is translated to Vulkan so that the capture layer -- which is a
/// Vulkan layer -- sees the frames of a game that draws in GL. A game whose GL
/// reached a native driver would render correctly and be captured as nothing,
/// which is the worst shape a failure can have here.
///
/// But the loader picks a driver by the *kernel device's* name. It looks for
/// one called `virtio_gpu`, finds that the only driver built is `zink`, and
/// gives up with `virtio_gpu: driver missing`. It does not fall back, and
/// `zink` is never chosen for an arbitrary device on its own. So it is named.
///
/// Measured 2026-09-12: without these, every process that touched the GPU
/// failed to create an EGL screen, in a box whose Vulkan drivers were both
/// present and loadable.
const GRAPHICS: &[(&str, &str)] = &[
("MESA_LOADER_DRIVER_OVERRIDE", "zink"),
("GALLIUM_DRIVER", "zink"),
// For anything that goes through libglvnd. Harmless where nothing does.
("__GLX_VENDOR_LIBRARY_NAME", "mesa"),
// **Intel's Vulkan Video is off unless asked for.** Its driver gates the
// video encode and decode extensions behind this, so on an Intel host the
// capture layer finds no encode support, produces nothing, and says
// nothing about why -- a box that streams a black screen while every
// component reports success.
//
// Read only by Intel's driver, so it costs nothing on a host with any
// other GPU. Measured 2026-09-12 on an Arc A310: without it, capture
// produced no output at all.
("ANV_DEBUG", "video-encode,video-decode"),
// **Audio is not under this user's runtime directory.** The services that
// serve it run as somebody else, so the socket lives somewhere both can
// reach and both are told where. Without this a game renders and plays
// silently, having looked under its own uid and found nothing.
("PIPEWIRE_RUNTIME_DIR", crate::services::AUDIO_DIR),
];
/// Mount one share where the descriptor says to put it.
///
/// The tag names an export; nothing here is a path on the other side of the
@@ -263,6 +371,103 @@ fn failed(share: &Mount, error: io::Error) -> Failure {
mod tests {
use super::*;
/// The driver override has to reach the workload, because nothing else
/// carries it: the image's profile script never runs for an exec'd
/// process. Without it a game's OpenGL finds no driver at all.
fn exec_with(env: &[(&str, &str)]) -> Exec {
Exec {
argv: vec!["/bin/true".into()],
env: env
.iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect(),
cwd: None,
uid: 1001,
gid: 1001,
}
}
/// The override has to reach the launch, and asserting that it is in a
/// table is not asserting that. A commit once defined the table and never
/// applied it; the tests passed and a dead-code warning was the only sign.
#[test]
fn the_launch_is_told_which_gallium_driver_to_use() {
let env = environment(&exec_with(&[]), None);
let driver = env
.iter()
.find(|(k, _)| k == "MESA_LOADER_DRIVER_OVERRIDE")
.map(|(_, v)| v.as_str());
assert_eq!(
driver,
Some("zink"),
"without this a game's GL reaches a native driver, renders \
correctly, and is captured as nothing"
);
}
/// Intel's driver hides Vulkan Video behind a debug variable, and the
/// capture layer needs video encode.
///
/// Without it the layer loads, finds no encode support, produces nothing,
/// and reports nothing -- so the box streams a black screen while every
/// component says it is working. It cost an evening to find once.
#[test]
fn intels_vulkan_video_is_asked_for() {
let env = environment(&exec_with(&[]), None);
let debug = env
.iter()
.find(|(k, _)| k == "ANV_DEBUG")
.map(|(_, v)| v.as_str())
.unwrap_or_default();
assert!(
debug.contains("video-encode"),
"on an Intel host this is the difference between a stream and a \
black screen, and neither says which: {debug:?}"
);
}
/// The directory made for the launch has to be named to the launch.
///
/// Making it and saying nothing is indistinguishable from not making it:
/// the environment is cleared, so a workload inherits no path to it. The
/// compositor in this image falls back to `/tmp` rather than failing, which
/// means the whole session comes up, works, and puts one user's sockets in
/// a directory every other user can write. ref(d-0065)
#[test]
fn the_launch_is_told_where_its_runtime_directory_is() {
let env = environment(&exec_with(&[]), Some("/run/user/1001"));
let runtime = env
.iter()
.find(|(k, _)| k == "XDG_RUNTIME_DIR")
.map(|(_, v)| v.as_str());
assert_eq!(runtime, Some("/run/user/1001"));
}
/// A directory that could not be made is not claimed to exist.
///
/// Pointing a workload at a path that is not there is worse than leaving it
/// unset: unset is a case every toolkit handles, and a bad path is one they
/// report as something else.
#[test]
fn a_launch_without_a_runtime_directory_is_told_nothing() {
let env = environment(&exec_with(&[]), None);
assert!(!env.iter().any(|(k, _)| k == "XDG_RUNTIME_DIR"));
}
/// Last wins, so a host can override what the image assumes.
#[test]
fn the_callers_own_environment_beats_the_images() {
let env = environment(&exec_with(&[("GALLIUM_DRIVER", "something-else")]), None);
let chosen: Vec<&str> = env
.iter()
.filter(|(k, _)| k == "GALLIUM_DRIVER")
.map(|(_, v)| v.as_str())
.collect();
// Both are present; `envs` applies in order, so the last is the one
// the process gets.
assert_eq!(chosen.last(), Some(&"something-else"));
}
fn share(ro: bool) -> Mount {
Mount {
tag: "user".into(),

82
apps/nesinit/tests/services_stop.rs Normal file
View File

@@ -0,0 +1,82 @@
// What the service stack does with its children when it goes away, against
// real processes.
//
// Its own test binary for the same reason as `reaping`: these wait on children,
// and a reaper in another test in the same binary would collect them.
use std::time::{Duration, Instant};
use nesinit::reap::Waiters;
use nesinit::services::{Service, Services, Stack};
/// A service that stays up until something stops it, and one that binds a
/// socket -- which is all the table needs to be for either question here.
static SLEEPERS: &[Service] = &[
Service {
name: "sleeper",
argv: &["/bin/sleep", "60"],
env: &[],
user: None,
cost: "nothing: this is a test",
required: true,
umask: None,
ready: None,
},
Service {
name: "second-sleeper",
argv: &["/bin/sleep", "60"],
env: &[],
user: None,
cost: "nothing: this is a test",
required: true,
umask: None,
ready: None,
},
];
/// Whether a pid is still a live process, asked without reaping it.
fn alive(pid: i32) -> bool {
// Signal 0 checks for the process without sending anything.
unsafe { libc::kill(pid, 0) == 0 }
}
/// Dropping the stack stops what it started.
///
/// As PID 1 the ordered shutdown would reach these anyway. Run by hand -- which
/// is how a guest that will not boot is debugged -- nothing else does, and the
/// bus, the audio server and the hub were left running with sockets nobody was
/// serving.
#[tokio::test]
async fn a_stack_that_goes_away_takes_its_services_with_it() {
let waiters = Waiters::new();
let mut stack = Stack::from_table(waiters, SLEEPERS);
let up = stack.bring_up().expect("two sleeps did not start");
assert_eq!(up.len(), 2);
let pids = stack.pids();
assert_eq!(pids.len(), 2, "the stack did not keep what it started");
assert!(pids.iter().all(|&pid| alive(pid)));
drop(stack);
// Signalled, not waited for: the stack cannot reap on its way out, so what
// is asserted is that each one leaves, not how fast.
let deadline = Instant::now() + Duration::from_secs(5);
for pid in pids {
loop {
// Nothing here reaps, so a signalled child becomes a zombie rather
// than disappearing -- and a zombie still answers signal 0. It is
// waited for explicitly instead.
let mut status = 0;
let seen = unsafe { libc::waitpid(pid, &mut status, libc::WNOHANG) };
if seen == pid || seen == -1 {
break;
}
assert!(
Instant::now() < deadline,
"{pid} was still running five seconds after its stack was dropped"
);
tokio::time::sleep(Duration::from_millis(20)).await;
}
}
}