There was a cap on how much this would read and none on how long it would
wait. The far end can accept a connection and then write nothing, and a
read with no deadline turns that into a poll loop that never runs again:
the address already forwarded stays correct, and the better one that
arrives afterwards is never seen. That is the failure this polls to avoid,
reached by a different route.
Five seconds, against a two second interval, so an answer that is merely
slow still lands and one that is never coming is abandoned.
The test hangs against the code as it was, which is the whole point of it.
The root arrives read-only and this process is PID 1, so until it mounts
them there is no /proc and nowhere in the filesystem to put a socket.
Nothing else in the guest is an init system, so nothing else was going to.
The symptom was three failures that look unrelated and share one cause. On
a real box the payload relay could not bind, with EROFS; whatever serves
the session's address could not bind either, the same way; and this process
could not make itself ineligible for the OOM killer, because /proc was not
there to write to. What the caller saw was a workload that ran and
published nothing, which is true and says nothing about why.
/proc is mounted first and unconditionally: finding out what an image
already mounted requires it, and it is therefore the one entry that cannot
be checked that way itself. Everything after it is skipped when it is
already present, so an image that does this properly is not mounted over.
Failures warn rather than abort. Refusing to boot would replace a session
that fails with a reason by a guest that never dialled out at all, and the
second is harder to diagnose from the outside.
The relay's directory is named by the module that owns the socket rather
than spelled again here, with a test tying the two together: a rename that
reached one and not the other would put the relay back exactly as it was.
Whatever serves media in a box knows how it can be reached, and the person who
needs to know is not in the box. Standard output here is a log file inside a
VM, so the control channel is the delivery path rather than a convenience --
which makes this init's job and not a detail of whichever component happens to
bind the port. The socket it reads was already documented as being read this
way; nothing read it.
Polled rather than read once, because an address is not a value but the best
answer so far. An endpoint discovers more ways to reach it after it binds, so
the first answer is the one that works on a local network and fails from
anywhere else. Only a changed answer is forwarded.
It dials rather than listens, which is the opposite of the relay next door and
deliberate: there the guest listens because the workload starts later, and here
the server is the long-lived one. Dialling also makes a server that has not
bound yet something to retry rather than something to wait for without knowing
whether it is coming.
The address itself is never logged. It is a capability to reach the session,
and a log inside the guest is the one place it has no reason to be.
A carrier that stops does not end a session: whatever was already reported is
still correct, and the workload's exit still has to be.
Three problems in the relay, all of them found in review.
Handing an envelope over waited for room. That loop also carries stop,
shutdown and the workload's exit, so a workload slow to read its own mail — or
one that never connected — could hold the lifecycle layer still behind it. It
never waits now: an envelope that will not fit is dropped, which costs nothing,
because what crosses this layer is re-sent when it changes.
Envelopes were queued for a workload that was not there. The queue filled with
copies that would be stale by the time anyone connected, and filling it was
what stalled the session. Nothing is held while the socket has nobody on it.
A frame had no maximum length. The workload can write for as long as it likes
without ever sending a newline, and the process assembling that is the one the
kernel has been told not to kill, so the memory it takes comes out of
everything else in the guest. Past 64 KiB the connection is dropped and the
relay waits for the next one; the failure says how long the frame got and
nothing about what was in it.
Also, a tag or a mount point with a nul byte in it was quietly turned into an
empty string, so an unmountable descriptor arrived later as a mount failure
about something else, after the mount point had already been created. It is
refused by name now, before anything is created.
The relay's tests grew a harness that waits for the connection to be carried
before sending anything down it, because dropping what arrives with nobody
connected made "connected" something a test has to establish rather than
assume.
Builds on the previous change, which had PID 1, the channel and the trait but
mounted nothing.
The shares are mounted now: a tag names an export, the descriptor names where
it lands, and every share goes on nosuid and nodev whether or not it is
writable — a share is data handed to the guest, and no descriptor has a way to
ask for a setuid binary or a device node in one. Mounting needs privileges a
test does not have, so the arguments and flags are derived by a function the
tests can assert, which is where the read-only decision lives.
Progress is reported in two messages rather than one. A share that did not
mount and a command that did not run are not the same incident, and each
carries the reason the operating system gave and the path it happened on: a
permission error on a named directory can be acted on, where "the share did not
mount" cannot.
The second layer is relayed and never read. Bytes arrive on the channel in an
envelope, cross a unix socket to the workload, and come back the same way. The
body is a string rather than nested JSON on purpose: a document this component
can index into is a document it can grow to depend on, and then the layer is no
longer opaque and the boundary it exists to draw is gone. An envelope is never
logged — not the body, not truncated, not at debug level — and the channel name
with a byte count is the whole of what may be said about one. The type's Debug
is written by hand for the same reason, because a derived one puts the body one
careless format string away from a log line.
A write to a channel nobody is reading now ends the session the same way a
closed read does. A caller that has stopped listening has also stopped being
able to say stop, which is one situation and was two outcomes.
The guest listens on the relay socket and the workload dials in, which is the
convention the other guest sockets already use and removes the startup ordering
problem: a workload that is not running yet has simply not connected yet.
A pid is only a name for a process until that process is reaped; after that
the kernel may hand the same number to something else. The handle kept the
number, so a stop or a kill issued during shutdown — which every session
outcome reaches — could land on a process nobody meant, and the one aimed at
the workload would have been a SIGKILL.
The registry now clears the flag as it delivers the exit, in the same call, and
nothing signals a pid whose flag is down. Waiting for the workload on the way
out takes the same answer: already reaped is already gone.
There is a window left, between the reap and the delivery, and closing it
entirely needs a handle the kernel keeps rather than a number. Recorded rather
than papered over.
Three problems in the shutdown and reaping paths, all of them found in review.
Reaping and waiting cannot be two mechanisms. `waitpid(-1, ...)` collects any
child, so the reaper and a caller waiting on its own child race for the same
status, and whichever loses gets nothing — losing the workload's exit, which is
the one thing this component exists to report. The reaper is now the only
waiter and hands each exit to whoever asked for that pid. Registering interest
holds the same lock the delivery takes, so a child that exits before its caller
is registered is still delivered rather than dropped; there is a test that
fails without that.
Killing the workload killed everything. `kill(-1, SIGKILL)` is every process
init may signal, so a workload that overstayed its grace period took the
guest's services with it, before the ordered stop the shutdown promises them
had even started. It signals the one pid now.
The shutdown had no workload to stop. It built a fresh handle with no pid, so
the graceful stop was a no-op and the workload only died in the sweep that
follows — which is exactly the order this was written to avoid. The handle the
session used is now the handle the shutdown uses, and waiting for the workload
waits for that pid rather than for any child to leave.
A microVM has no init unless something is it, and three of the jobs belong to
nothing else in the guest: reaping whatever the workload orphans, turning a
signal into an ordered shutdown, and being the guest end of the one channel
out.
None of it knows what it is running. The guest dials out on a fixed vsock port,
says its protocol version first, is handed one boot descriptor — a command
line, shares, output geometry, and what an exit means — and carries that out.
There is no code path that branches on which workload started, which is the
property the component exists to keep.
It reports and does not supervise. When the workload ends, the exit goes up the
channel and the session is over; `on_exit` says what the exit means, and
starting something again is a decision for the end that can see whether
restarting is repair or a loop. A signalled workload is reported as signalled
with no exit code, because reporting 0 for a killed process makes a kill look
like a clean run.
Two seams keep this testable without a VM, which is the reason for both of
them. Reaping runs against real forked children, with the subreaper bit making
a test process inherit orphans the way PID 1 does. The channel is generic over
the byte stream, so the exchange is driven over an in-memory pipe — the
transport contributes nothing to the protocol beyond ordering and framing.
The lifecycle types live in nesprotocol behind a feature, off by default: both
ends of the channel read one definition and cannot drift from it silently,
while the media components keep building without serde.
Mounting shares is not implemented in this build. The descriptor's mounts are
refused rather than ignored — a workload started without the shares it was
promised fails later, somewhere else, for a reason nobody can see from here.
