netris-nestri/apps/nesinit/README.md

nesinit

PID 1 inside a box.

A microVM has no init unless something is it, and in a box nothing else is: there is no service manager in the image and no init scripts. Four jobs, and this is all of them:

• Reaping. A process whose parent dies is reparented to PID 1. Without a reaper, every orphan the workload leaves behind holds a pid and a slot in the process table until the guest is gone.
• Ordered shutdown. The workload stops first and alone, then everything else, then the disks are flushed and the machine is powered off. An init that returns leaves a guest running with nothing in it.
• The box's own services. The bus, audio, and the transport that carries a session out, started in order from a table compiled into this binary. There is no unit format and no directory of files to read: the services in a box are fixed, and running on any distribution comes from depending on none of their init scripts rather than from being configurable.
• The guest end of the control channel. One vsock connection out, carrying what to run in and what happened back.

It does not know what it is running. It is handed a set of shares, and then commands naming what to run and what an exit means; there is no code path here that branches on which workload it started, and there is not meant to be.

A box outlives what runs in it. Init mounts, brings the services up, says it is ready, and then takes commands for as long as the box lives — so this image on its own runs nothing at all, and a box may be launched into more than once.

The channel

The guest dials out on a fixed vsock port and speaks first:

guest → { "type": "ready", "protocol_version": 3 }
guest ← { "type": "boot", "mounts": [...] }
guest → { "type": "mounted" }
guest → { "type": "initialized", "services": ["dbus-system", ...] }
guest ← { "type": "launch", "id": "…", "exec": {...}, "on_exit": {...} }
guest → { "type": "started", "id": "…" }
guest → { "type": "workload_exited", "id": "…", "exit_code": 0 }

ready is the handshake and initialized is the box working. They are two facts and must not be treated as one: a caller that waits on the first has a wait that succeeds before anything in the guest has started.

Every launch carries an id and every message about a launch carries it back. Without one, a second launch's exit is indistinguishable from the first's — which reads at the far end as a finished session still running, or a running one reported as stopped.

Newline-delimited JSON. Dialling out rather than being connected to is worth keeping for two reasons: the listener is up before the VM starts, so nothing races a booting kernel and nothing has to retry, and the connection establishing is itself the liveness signal — without it the far end needs a timeout to tell a slow boot from a dead one.

The version goes out before anything is read, so a peer that cannot talk to this build refuses it before handing over a descriptor rather than failing later on a field that turned out to be missing.

The types are in nesprotocol::lifecycle, behind the lifecycle feature, so both ends of the channel read one definition and neither can drift from it silently.

mounted / mount_failed stay separate from started / start_failed because the two want different things looked at: a share that did not appear and a command that did not run are not the same incident. A failure carries the reason in the words the operating system used, and the path it happened on — a permission error on a named directory can be acted on, where "the share did not mount" cannot.

Two layers, one channel

The channel carries a lifecycle layer, above, and a payload layer that nesinit relays and never reads:

{ "type": "payload", "channel": "<name>", "body": "<opaque string>" }

Both directions. Inside the guest an envelope crosses a unix socket at /nestri/payload.sock, which the guest listens on and the workload dials into. That socket is a mechanism and expected to change; the envelope is the boundary and is not.

Nothing is held for a workload that is not on the relay, and nothing waits on one that is slow to read. An envelope that arrives with nobody connected is dropped, and so is one that arrives faster than the workload reads: what crosses this layer is re-sent when it changes, so a queued copy is a stale copy — and the queue that would hold it is on the same loop that carries stop, shutdown and the workload's exit, none of which may wait behind it.

A frame is capped at 64 KiB. The workload is on the other end of that socket and can write for as long as it likes without ever sending a newline; the process assembling it is the one the kernel has been told not to kill, so an unbounded buffer there comes out of everything else in the guest. Past the cap the connection is dropped and the relay waits for the next one.

body is a string rather than nested JSON, deliberately. A document nesinit can index into is a document nesinit can grow to depend on, and then the layer is no longer opaque and the boundary it exists to draw is gone.

An envelope is never logged. Not the body, not truncated, not at debug level. The channel name and the byte count are the whole of what may be said about one — what crosses here includes credentials meant for the workload and nothing else. Payload's Debug is written by hand for the same reason: a derived one puts the body one careless {:?} away from a log line.

The shares

Each mounts entry is a tag, a path to put it at, and whether it is read-only. The tag names an export and is never a path on the other side of the channel, so the guest learns nothing about the filesystem it is handed a piece of. Choosing where a share lands is the descriptor's job, not the guest's: deciding that means knowing what the workload expects to find there, which is exactly the knowledge a workload-independent init does not have.

Every share is mounted nosuid and nodev, whether or not it is writable. A share is data handed to the guest, and no descriptor has a way to ask for a setuid binary or a device node in one.

uid and gid in exec are load-bearing rather than hygiene. Whoever writes the descriptor also exported the writable share, so the two have to agree; when they do not, the first write is refused and the failure surfaces here as a permission error with a path, instead of as a workload that misbehaves much later for no visible reason.

It reports; it does not supervise

When a launch ends, the exit goes up the channel. on_exit says what that exit means — whether it ends the session or leaves the box up to be launched into again — and nothing here restarts anything of its own accord. Starting something again is a decision for the end that can see whether restarting is repair or a loop.

restart exists as one message and is defined as exactly that: a kill followed by a launch of the same command, keeping the id, with no retry and no backoff. It is one message rather than two only because a caller sending two has the same effect with a worse race in it.

The same rule covers the box's own services. One that dies is reported and left dead — nothing else in the guest is watching them, so a death that is not said here is a box that looks healthy and cannot work.

One launch at a time. A launch arriving while one is running is refused, carrying the id it was asked for, rather than queued or silently replacing it: a box has one screen, so a second concurrent launch has nowhere to draw.

A signalled workload is reported as signalled, with no exit code. Reporting 0 for a killed process would make a kill look like a clean run.

What is not here yet

It has never been PID 1 of anything. Every line of this is written for a box and all of it has been tested outside one. It runs perfectly well as an ordinary process — it warns rather than fails when it is not PID 1 — which is how most of it is exercised, and is also how a guest that will not boot gets debugged: init=/bin/bash on the kernel command line, then run this by hand and watch it fail.

Output geometry is deliberately absent from this layer. The compositor is started by a launch, with that launch's geometry in its own arguments, so the numbers appear in one place rather than two that can disagree.

Testing

cargo test -p nesinit

No VM required, and that is the point of the seams. Reaping is tested against real forked children — PR_SET_CHILD_SUBREAPER makes a test process inherit orphans the same way PID 1 does. The channel is tested over an in-memory pipe, because the transport contributes nothing to the protocol beyond ordering and framing. The relay is tested over a real unix socket. Mounting needs privileges a test does not have, so what is asserted is the arguments and flags the mount is given, which is where the read-only and nosuid decisions live.