- **Installer:** piped into `sudo`, installs to `/usr/local/bin`, and
hands onboarding `/var/lib/nestri`. Root lets the host agent jail each
box under its own uid and give guests a network. A host that ran the
agent as the sudo user has that state copied over (never moved), so it
keeps its machine identity and its downloads. Pins host agent 0.2.4.
- **Base image:** `vhost-user-nvgpu` and `nvgpu-userspace`, static
(musl), from the same `NVGPU_REF` as the guest driver and Vulkan layer,
under `/boot/host`. The host agent takes them out at unpack, as it does
the kernel, and forwards the GPU when it finds the NVIDIA driver loaded.
Merge after the host agent 0.2.4 release: the installer pins it.
Re-running the install command on a host that runs the agent failed:
onboarding refuses to race the agent holding the host lock. The
installer now stops `nestri-host.service` first; onboarding starts it
again on the new binary.
A `kernel-build` stage runs `kernel-build.sh` and puts the result at
`/boot/vmlinux`; host onboarding takes it out beside the root image, so
kernel and image (and their shared NVGPU_REF) always ship as one pair.
`make push` publishes `:latest` and a commit tag from the build box.
Built on the build box: `/boot/vmlinux` is 24 MB, and
`vmlinux.nvgpu-rev` is beside it.
A QR sign-in on Nestri's own hardware proves the Steam account, so `POST
/machine/enrolment` from a machine in `FLEET_ORGANISATION_ID` also links
it to the person. Hosts somebody brought never link (a machine vouching
for itself could attach any Steam account to anyone enrolled on it). A
Steam account already linked elsewhere stays there; the report is
recorded either way.
Tests: three new in `enrolment.test.ts` (fleet links and is idempotent,
own host does not link, taken account is left alone). 164 pass.
The guest's audio stream ran about 3.5% short of realtime: 193 packets a
second where 200 were due, with no network loss at all. The client plays
at its own 48 kHz, so its buffer drained and underran every few seconds,
idle or loaded.
The graph was running 128-frame (2.67 ms) cycles. The 256 floor for VMs
matched on `cpu.vm.name`, which PipeWire reads from DMI and nesbox does
not provide, so it never applied, and neswire's 240-frame request was
rounded down to a power of two. A busy guest missed ~20 of those cycles a
second (pw-top ERR on neswire), and each missed cycle's audio never
reached neswire while the graph clock moved on.
- PipeWire is fixed at 240 frames, power-of-two rounding off: one cycle,
one 5 ms Opus packet, twice the headroom of before.
- neswire reads the graph position each cycle and fills exactly the
frames a missed cycle skipped, as silence, so the stream stays at
48 kHz whatever the scheduler does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each client states what it decodes when it connects, and the hub passed
each statement to the encoder as it arrived. The newest client decided, so
a client that only decodes H.264 lost its picture as soon as one that
decodes AV1 or H.265 joined.
The hub now keeps every client's statement and gives the encoder their
intersection, recomputed when a client joins, when one leaves, and when the
controller returns to automatic. A room settles on the best codec all of
its clients share, and moves up again once the client holding it back has
left. Manual mode still leaves the encoder alone; the change is applied
when the mode goes back to automatic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A Steam account is linked only through Steam's own OpenID sign-in
(POST /steam/link/start, GET /steam/link/callback). POST /steam/link,
which linked any Steam id on trust, is removed. Migration 0020,
additive.
- The installer installs host agent 0.2.2 (virtiofsd preflight, install
wording), released from nescore prod.
POST /steam/link took a Steam id from the request body and linked it. A
Steam id is public, so anyone could claim any account, its library, and
the one free weekend that is keyed on it.
Linking is now POST /steam/link/start, which returns a Steam sign-in URL
tied to a one-time, ten-minute request, and GET /steam/link/callback,
where the browser comes back. The callback accepts the id only if Steam
confirms the assertion, the assertion was issued for exactly this
callback, and the id is read from Steam's claimed identity URL in the
one shape Steam issues it. The request is spent on use.
One line: the installer's default host agent version is now 0.2.1,
released from prod. It ends a run when the job poll sends session.stop,
which the free weekend relies on.
Ships everything on dev to prod, including:
- POST /box: create a box on your own host or on the fleet
- Fleet entitlement during your own run; fleet runs billed to the box
owner's personal team
- The free weekend trial: Fri–Sun, 2 hours, one per team, email and
Steam account, off once five teams pay; session.stop jobs for overdue
trial runs
- Install tokens that register a host to an organisation's fleet
- Migrations 0018 and 0019, both additive
Plus the sign-in screen work, the guest image changes and the team/box
reads already on dev.
Fleet placement stays off until FLEET_ORGANISATION_ID is set on the API
host.
An install token can now name an organisation instead of a team, and the
host it registers belongs to that organisation's fleet. Only members of
the organisation may mint one, since fleet hardware is rented to other
people. A token for both is refused, by the route and by a check
constraint, as it already is for a machine.
A team with no plan may run on the fleet from Friday 00:00 to Monday
00:00, Helsinki time, for two hours of live play per weekend. One trial
per team, per email address and per Steam account; the claim outlives
the account, so closing it and signing up again is not a second trial.
Once five teams pay, the trial closes for everyone.
A trial run is marked on the session. Its time is counted from the run's
own live interval and kept out of the plan's allowance. Unlike an
allowance, the trial ends a run: the host's work poll now carries a
session.stop job for any trial run past its hours or its window, which
an agent that does not know the kind already skips.
A fleet machine belongs to an organisation, which is not something that
pays, so fleet runs were neither checked nor metered. The payer is now
the box owner's personal team, and the meter that starts and stops with
a run asks the same function the start check does, so the team checked
and the team billed cannot differ.
Fleet machines refused everyone, because nothing could yet say who had
paid for them. A run on one can only exist once it passed the start
check, so a starting or live run of your own box on that machine is now
what grants access, and it ends with the run. Holding a box is not
enough: a box costs nothing to hold.
POST /box places a box once, at creation. On your own host it keeps the
existing rule (exactly one host, or refuse). On the fleet it picks an
online machine owned by the fleet organisation, the one holding the
fewest boxes. Each person holds at most one fleet box. Which organisation
is the fleet is deploy configuration, so no deployment rents hardware it
was not told to.
No machine online is a 429, since the same request succeeds once one is
back.
send_many_datagrams queues a frame's fragments under one lock hold and
one driver wake instead of one per fragment. It also rejects the whole
frame on TooLarge, so a path that shrinks mid-frame no longer leaves half
a frame on the wire for the client to time out. Eviction behaviour is
unchanged. A refused PyroWave frame now keeps its sequence number, since
nothing went out under it. pyro_bench mirrors both ends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On an NVIDIA host, NVIDIA's Vulkan driver in the guest lists ray
tracing, optical flow, binary import and CUDA kernel launch, then fails
`vkCreateDevice` with `VK_ERROR_INITIALIZATION_FAILED` for each of them.
An engine that enables what it is offered does not start. virtio-nvgpu's
guest layer (`vklayer/`, nestrilabs/virtio-nvgpu#5) hides those
extensions. It acts only on an NVIDIA device in a guest that has the
virtio-nvgpu driver, so on AMD and Intel boxes it loads and does
nothing.
- A new `nvgpu-layer-build` stage builds the layer from virtio-nvgpu at
`NVGPU_REF`, the same setting the kernel build uses for the guest
driver. The commit is recorded in `/usr/share/nestri/virtio-nvgpu.rev`.
- The layer and its manifest go in `/usr/lib` and
`/usr/share/vulkan/implicit_layer.d`. The missing-library check covers
the layer.
**Merge after nestrilabs/virtio-nvgpu#5.** The default `NVGPU_REF=dev`
has no `vklayer/` until that PR lands, so `make build` fails at this
stage before then.
Built on nestripc-1 with `NVGPU_REF=feat/v0.2-compute` (virtio-nvgpu
112eac4) and pushed:
- `ghcr.io/nestrilabs/nestri/base:latest` sha256:152a4485…
- `ghcr.io/nestrilabs/nesbox/jail:latest` sha256:df813051…, rebuilt on
that base from nesbox dev 6d37597, which includes the vhost-user
forwarding
Measured on box2 (RTX 3060, 595.104.02) with virtio-nvgpu's vkrt probe:
with the layer on, a plain device is created and none of the seven
extensions are offered. With it off, all seven are offered and fail with
-3.
On an NVIDIA host the guest's Vulkan driver offers ray tracing, optical
flow and a few others it cannot create a device with, and an engine that
enables them does not start. virtio-nvgpu's layer hides them; it acts
only on a forwarded NVIDIA device, so AMD and Intel boxes are unaffected.
Built from virtio-nvgpu at NVGPU_REF, the ref the kernel build already
takes the guest driver from, and recorded in the image.
The end that reads the first frame of an attach stream to learn whom the
connection is for then forwards the rest untouched, and what arrived behind that
frame in the same read belongs to the stream.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The host can now ask a box for a shell on a pseudo-terminal, the way a container
runtime attaches to a container. `HostToGuest::Attach` starts the process, and
the guest then dials the host on a vsock port of its own (7001) and carries the
terminal's bytes over a small framed stream: data both ways, a resize, a hangup,
and the exit status last. A screenful of output on the JSON-lines control channel
would hold every other message in the box behind it, which is why it is not
there. A failure to start is the exception, and is said on the control channel
because nothing has connected to the data port yet.
A shell is not a launch: it never ends the box, and the box ending ends it. It
is started through the same reaper as everything else, so it cannot become the
zombie that exists to be prevented. With a terminal on the other end the PTY is
ordinary; without one it is put in raw mode so a command run for its output gets
its bytes back unrewritten. Closing the stream hangs the shell up, process group
and all.
The frame codec is a pure bytes-in, frames-out module in nesprotocol with a
payload cap and no skipping of unknown tags, since a length read from a guest is
read across a trust boundary. No version bump: adding a message does not need
one, and a guest that predates it simply never dials.
The runtime stage gains strace, lsof and less, so a root shell in a box is not
one that cannot trace a process.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There is no service manager in a box, so nothing applies the distribution's
sysctl configuration, and the guest ran on the kernel's own defaults. Two of
them are too low for a modern engine under Proton:
- vm.max_map_count defaults to 65530. A large Unreal Engine 5 title maps far
more regions than that, and the failure is a null allocation deep inside the
engine or Wine rather than a message about mappings. It is set to 2147483642,
the value SteamOS and the distributions that package Proton use. It is read
back after the write, because an accepted write that did not take reports
success while the box is still unconfigured.
- RLIMIT_NOFILE is raised on init, which every process in the box inherits. A
child can lower a limit but cannot raise its hard one, so the unprivileged
workload could never get above what it started with. 524288 is the hard limit
Proton's documentation asks for. It is only ever raised.
Both are best effort and each failure says what it costs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The distribution's rust package is built against the distribution's LLVM, and
when the two repositories this image draws from disagree about llvm-libs, rustc
itself fails to start with a symbol-lookup error, before any of our code is
involved and in every stage that builds Rust. rustup's toolchain links nothing
of the distribution's, so `rustup default stable` replaces the package.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`umu-run` is how a title that is not Steam's is launched, so the game fixes
Proton carries apply outside Steam. It is built from a pinned tag in its own
stage rather than installed from the distribution, whose package declares a
long list of lib32 libraries and desktop packages for the Steam Linux Runtime
container this image never runs.
It is built against the distribution's urllib3 and pyzstd instead of the copies
it can vendor: those are built from source with pinned build tooling, and a
rolling distribution moved past a pin (setuptools-scm 10 against a <10
ceiling) and failed the build. The image adds python-xlib, python-urllib3 and
python-pyzstd, none of which pull in anything 32-bit.
Two checks keep it honest. umu-run has to start, because a missing Python module
otherwise shows up as a game that never starts and reads like a script failure;
and no lib32 package may be installed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Polishes the sign-in screens and adds a way to see all of them locally.
**Screens**
- Code fields are drawn as one box per character (device code shows a
dash between halves). One real input sits under the boxes, so typing,
paste and autofill work normally, and with script off it falls back to
the plain field.
- A wrong answer or a refused character shakes the field and rings it
red.
- Focus is white instead of the brand colour; the button stays dim until
the form can be sent.
- One alert at a time: an error replaces "Code sent to …" instead of
joining it.
- The device confirm screen says "A device" instead of the client's
self-chosen name, with the Deny warning moved under the buttons.
- A device code typed with its dash (`BCDF-3467`) is no longer cut to
`BCDF-346`.
**Preview**
`bun run preview` in `apps/auth` serves all 18 screen states on :1338,
at desktop and phone sizes, reloading on save. No database or mail
needed.
To make that possible the issuer's screens moved out of the route
handlers into `packages/auth/src/ui/issuer.ts`, unchanged.
Tests: `bun test` in `packages/auth` (67 pass).
A code field silently dropped a character it does not accept, so typing
a letter into the emailed code, which is digits only, looked exactly
like a field that does not work. It now shakes the boxes and rings them
red, the same answer a wrong code gets. Dashes and spaces are still
taken quietly, since they are how a code is written.
The shake an error page plays on load now rings the field red too,
fading back to its usual border once it settles, on every field
including the email address. Reduced motion keeps the red and drops
the movement, since the red is the message.
The boxes were one input each, with script moving focus between them.
That worked in a clean browser and failed in an ordinary one: on the
emailed code, whose first box is marked as a one-time-code field, typing
and pasting did nothing, while the device code, which is not marked,
worked. A password manager or autofill prompt hooking that field and a
script moving focus between six fields were working against each other.
It also left gaps: pasting into the third box with the second one empty
produced a code with a character missing.
Now the real input is stretched invisibly over the boxes and is the only
thing typed into, pasted into or autofilled; the boxes only display what
it holds, with the next one ringed and showing a caret. With nothing
moving focus there is nothing to fight, and a code cannot have a gap.
The browser's own validation message works again too, since the field
it points at is where the boxes are.
Wanjohi
DatCaptainHorseandClaude Opus 5.5