Four findings from review, all of them real.
The relay's directory was mounted on the tree a session's shares live in. A
fresh tmpfs there hides every directory the image prepared underneath it: the
install, the user state, the work directory, and the mount point the log share
is attached to from fstab. A box would have come up with a socket and without
any of the places its workload looks for its files, and the exact-path check
could not notice, because what fstab mounts is a directory inside that tree
rather than the tree itself. It moves to /run, which is where a runtime socket
belongs, is a tmpfs already, and has nothing else mounted inside it.
It was also owned by this process and closed to everyone else, which stopped
the workload traversing it to reach the relay at all. The directory is now
readable and searchable, and still writable by nothing but this process, which
is what makes the socket in it unreplaceable; the socket itself is what the
workload is allowed to connect to. The permission belongs on the socket rather
than on the path.
The address served to a reader was built once at startup and served forever, so
a reader that polls for a better one could only ever get the first. An endpoint
does not know all of its own addresses when it binds: the first is the one that
works on the same network and fails from anywhere else. It is now rebuilt per
read, which is what makes polling for it worth doing.
And the address was taken from whoever held a path in a directory the workload
can write. Workload code could unlink the socket a service was listening on,
bind its own, and every read afterwards would hand the client an address of its
choosing -- a session given to somebody else rather than a session that fails.
The peer's credentials are now checked before a byte is read, from the kernel
rather than from anything the peer says about itself, and an address served by
the workload's own user is refused and said loudly.
That check is only worth something while the workload has a user of its own, so
the image grows one. Two users, and they must stay two: one runs the services
that ship in the image, the other is who a workload runs as. Sharing one does
not weaken the check, it makes every session fail it.
A workload running as root is every user at once and cannot be told apart from
anything; the check stands down there and says so at boot instead, because
refusing root would refuse whatever legitimately serves the address as well.
Also bumps tinyvec by a patch release. It does not build on this toolchain --
`vec` resolves to the module and not the macro -- which made every crate that
depends on an endpoint, including this one, unbuildable. Pre-existing and
nothing to do with this change; the lockfile said the same version before it.
The root arrives read-only and this process is PID 1, so until it mounts
them there is no /proc and nowhere in the filesystem to put a socket.
Nothing else in the guest is an init system, so nothing else was going to.
The symptom was three failures that look unrelated and share one cause. On
a real box the payload relay could not bind, with EROFS; whatever serves
the session's address could not bind either, the same way; and this process
could not make itself ineligible for the OOM killer, because /proc was not
there to write to. What the caller saw was a workload that ran and
published nothing, which is true and says nothing about why.
/proc is mounted first and unconditionally: finding out what an image
already mounted requires it, and it is therefore the one entry that cannot
be checked that way itself. Everything after it is skipped when it is
already present, so an image that does this properly is not mounted over.
Failures warn rather than abort. Refusing to boot would replace a session
that fails with a reason by a guest that never dialled out at all, and the
second is harder to diagnose from the outside.
The relay's directory is named by the module that owns the socket rather
than spelled again here, with a test tying the two together: a rename that
reached one and not the other would put the relay back exactly as it was.
Whatever serves media in a box knows how it can be reached, and the person who
needs to know is not in the box. Standard output here is a log file inside a
VM, so the control channel is the delivery path rather than a convenience --
which makes this init's job and not a detail of whichever component happens to
bind the port. The socket it reads was already documented as being read this
way; nothing read it.
Polled rather than read once, because an address is not a value but the best
answer so far. An endpoint discovers more ways to reach it after it binds, so
the first answer is the one that works on a local network and fails from
anywhere else. Only a changed answer is forwarded.
It dials rather than listens, which is the opposite of the relay next door and
deliberate: there the guest listens because the workload starts later, and here
the server is the long-lived one. Dialling also makes a server that has not
bound yet something to retry rather than something to wait for without knowing
whether it is coming.
The address itself is never logged. It is a capability to reach the session,
and a log inside the guest is the one place it has no reason to be.
A carrier that stops does not end a session: whatever was already reported is
still correct, and the workload's exit still has to be.
Builds on the previous change, which had PID 1, the channel and the trait but
mounted nothing.
The shares are mounted now: a tag names an export, the descriptor names where
it lands, and every share goes on nosuid and nodev whether or not it is
writable — a share is data handed to the guest, and no descriptor has a way to
ask for a setuid binary or a device node in one. Mounting needs privileges a
test does not have, so the arguments and flags are derived by a function the
tests can assert, which is where the read-only decision lives.
Progress is reported in two messages rather than one. A share that did not
mount and a command that did not run are not the same incident, and each
carries the reason the operating system gave and the path it happened on: a
permission error on a named directory can be acted on, where "the share did not
mount" cannot.
The second layer is relayed and never read. Bytes arrive on the channel in an
envelope, cross a unix socket to the workload, and come back the same way. The
body is a string rather than nested JSON on purpose: a document this component
can index into is a document it can grow to depend on, and then the layer is no
longer opaque and the boundary it exists to draw is gone. An envelope is never
logged — not the body, not truncated, not at debug level — and the channel name
with a byte count is the whole of what may be said about one. The type's Debug
is written by hand for the same reason, because a derived one puts the body one
careless format string away from a log line.
A write to a channel nobody is reading now ends the session the same way a
closed read does. A caller that has stopped listening has also stopped being
able to say stop, which is one situation and was two outcomes.
The guest listens on the relay socket and the workload dials in, which is the
convention the other guest sockets already use and removes the startup ordering
problem: a workload that is not running yet has simply not connected yet.
Three problems in the shutdown and reaping paths, all of them found in review.
Reaping and waiting cannot be two mechanisms. `waitpid(-1, ...)` collects any
child, so the reaper and a caller waiting on its own child race for the same
status, and whichever loses gets nothing — losing the workload's exit, which is
the one thing this component exists to report. The reaper is now the only
waiter and hands each exit to whoever asked for that pid. Registering interest
holds the same lock the delivery takes, so a child that exits before its caller
is registered is still delivered rather than dropped; there is a test that
fails without that.
Killing the workload killed everything. `kill(-1, SIGKILL)` is every process
init may signal, so a workload that overstayed its grace period took the
guest's services with it, before the ordered stop the shutdown promises them
had even started. It signals the one pid now.
The shutdown had no workload to stop. It built a fresh handle with no pid, so
the graceful stop was a no-op and the workload only died in the sweep that
follows — which is exactly the order this was written to avoid. The handle the
session used is now the handle the shutdown uses, and waiting for the workload
waits for that pid rather than for any child to leave.
A microVM has no init unless something is it, and three of the jobs belong to
nothing else in the guest: reaping whatever the workload orphans, turning a
signal into an ordered shutdown, and being the guest end of the one channel
out.
None of it knows what it is running. The guest dials out on a fixed vsock port,
says its protocol version first, is handed one boot descriptor — a command
line, shares, output geometry, and what an exit means — and carries that out.
There is no code path that branches on which workload started, which is the
property the component exists to keep.
It reports and does not supervise. When the workload ends, the exit goes up the
channel and the session is over; `on_exit` says what the exit means, and
starting something again is a decision for the end that can see whether
restarting is repair or a loop. A signalled workload is reported as signalled
with no exit code, because reporting 0 for a killed process makes a kill look
like a clean run.
Two seams keep this testable without a VM, which is the reason for both of
them. Reaping runs against real forked children, with the subreaper bit making
a test process inherit orphans the way PID 1 does. The channel is generic over
the byte stream, so the exchange is driven over an in-memory pipe — the
transport contributes nothing to the protocol beyond ordering and framing.
The lifecycle types live in nesprotocol behind a feature, off by default: both
ends of the channel read one definition and cannot drift from it silently,
while the media components keep building without serde.
Mounting shares is not implemented in this build. The descriptor's mounts are
refused rather than ignored — a workload started without the shares it was
promised fails later, somewhere else, for a reason nobody can see from here.
