All of dev to prod. Prod's squash copies are merged in with dev's
version kept, so the tree is identical to dev.
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- **Sessions (#412):** `end_reason` and `time_claimed`, with migration
`0026_session_end_reason` (two nullable columns, readable by the release
it replaces). `POST /session/:id/state` takes an optional `endReason`,
and a run stopped before it went live can now go from `starting` to
`ended`.
- **Installer (#411):**
- `bash <(curl -fsSL https://api.nestri.io/install.sh) -u` updates a
host; `-h` prints help; a plain token installs.
- When the script is not root, it asks sudo itself.
- The installed host agent version is unchanged (0.2.18).
**Ships before host agent 0.2.19**, which sends `endReason` and the new
transition.
Cut from prod with dev's files, so squash histories cannot conflict; the
tree is identical to dev.
- **Guest image (#405):** carries `libvpx`, `speex`, `libtheora`,
`libvdpau` and `jsoncpp`, which Proton's GStreamer plugins link. Without
them, `libgstlibav` and `libgstspeex` failed to load in every box on
every Proton shipped so far. Verified: every library both published
Protons need is now provided.
- **Billing (#406):** every Polar webhook logs its type, the team it
resolved to, the plan and status, whether it changed anything, and the
subscription and customer ids. Logging only.
Cut from prod rather than opened from dev: after squash merges the two
histories conflict, though the files differ only in these four. This
branch's tree is identical to dev's.
After merge, the guest image is published from this prod commit. Host
agent 0.2.18 then makes hosts pull it.
New installs get host agent 0.2.17. It is published at
/install/host/0.2.17/; the binary is checksum-verified and reports
`neslet 0.2.17`. 0.2.17 raises LAYOUT to 6, so hosts rebuild their guest
from the current image (OpenGL frontends, nescope answering every
frame).
Guest image fixes from #401, plus the dev changes that had not reached
prod yet.
- **OpenGL games render** (Dome Keeper, confirmed on an AMD host). The
guest's Mesa gains its EGL/GLX frontends, and `vm.overcommit_memory=1`
stops a large reservation being refused before it is used.
- **Chromium / NW.js games get past their first frame** (Sheepy).
nescope answers every commit, and paces Xwayland windows too.
- **Proton patch step.** `build/patches/proton/` and the `-nN` tag.
Nothing uses it until hosts are pointed at the new tag.
- Also included: the pixelforge update already on dev. It is in the
image tested today.
**Versions kept, not downgraded.** Dev still had host agent 0.2.13 and
desktop 0.1.0. The last commit restores prod's 0.2.16 and 0.1.2, so this
merge changes no install versions.
**After merge.** `make push` from this prod commit publishes the guest
image. The host agent release that raises LAYOUT to 6 makes hosts pull
it.
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
The real change. #399 merged by mistake without it: a rewritten copy of
the #398 merge commit, same tree, so nothing in prod changed. New hosts
install 0.2.16 of the host agent: onboarding retries a failed phase, and
finds steamclient.dll where Valve's client package now keeps it.
A customer the payment provider already knew under the same email kept
their old customer record, without our team id, so their paid
subscription was acknowledged and applied to no team. Checkout now
carries the team in its metadata and the webhook falls back to it; a
paid delivery that matches no team is logged.
The provider reuses a customer it already has under the same email and
leaves our team id off it, so the subscription arrived with no team,
was acknowledged, and changed nothing: a paid plan never applied. The
checkout now carries the team in its metadata, which the subscription
keeps, and the webhook reads it there when the customer has none. A
paid delivery that still matches no team is logged instead of dropped
in silence.
(cherry picked from commit 1301ff0806)
New hosts install host agent 0.2.15. It refuses NVIDIA's proprietary
kernel module at onboarding, since GPU forwarding needs the open kernel
modules (Turing or newer).
- A Steam sign-in on a person's own host now links the Steam account, as
it already did on ours. Before this, their first launch asked them to
link Steam with nothing in the app to do it from.
- Downloads hand out desktop app 0.1.1: a launch on a person's own host
makes its box instead of stopping at "You do not have a box yet".
- New hosts install host agent 0.2.14, which checks for mkfs.erofs
before it starts setting up.
A sign-in on a person's own host recorded the enrolment but never
linked the Steam account, so their first launch asked them to link it
again with nothing in the app to do it from. The QR sign-in is the same
proof on their host as on ours.
(cherry picked from commit 24ad9f6b78)
`/download/windows` and `/download/linux` redirect to a one-minute
signed URL for the current desktop app (`desktop/<version>/` in the
releases bucket). `/desktop.sh` installs it on Linux for the calling
user, without sudo. The version handed out is a constant bumped by a
commit after each release. Tests: 180 pass.
/download/windows and /download/linux redirect to a one-minute signed URL
for the current desktop app, from the same private bucket as the host
agent, so a download page can link to them once and every download still
passes through the API. Which version they hand out is a constant bumped
by a commit after a release is published, as the host installer's is.
/desktop.sh installs it on Linux for the calling user: one binary in
~/.local/bin and a launcher entry, no root.
Two changes from dev:
- **Runs wait in line for a free slot on their host.** A host says on
its heartbeat how many runs it holds and is handed only that many; the
rest wait, paid before free-weekend, then first come first served.
Reading a waiting run returns its place in line. A waiting run nobody
has asked after for a minute leaves the line. A host that does not
report slots is handed every run, as before, so this is safe to deploy
ahead of the host agent.
- **Free days are Saturday, Sunday and Monday.**
Adds migration `0025_run_queue` (two nullable columns). API tests: 177
pass.
A host holds only so many runs at once, and until now every waiting run
was handed to it regardless; the ones it could not hold failed there.
The host now says on its heartbeat how many runs it holds, and is handed
only as many waiting runs as it has free. The rest wait in line: paid
runs before free-weekend runs, then first come first served. Reading a
waiting run returns its place in line, and says when a paid run that
asked later has gone ahead, so a free player is never reordered quietly.
A waiting run nobody has asked after for a minute was abandoned, and
leaves the line rather than holding up everyone behind it. A host that
does not report its slots is handed every run, as before.
Cherry-pick of aa824a1 from dev, alone.
`GET /box` gains `host` (the machine's DNS label) and `fleet` on listed
boxes. The desktop app uses them to open the fleet machine's Steam
sign-in during onboarding. Additive; nothing existing changes shape.
Tests: 172/172 against Postgres.
The desktop app gives a new cloud user a fleet box and then opens that
machine's Steam sign-in on its hostname; GET /box now says which hostname,
and which box is the fleet one.
Pins the installer to host agent 0.2.12. In this release:
- a run boots at the size it asked for;
- each run narrates its install progress through the edge;
- hosts re-pull the guest image and the jail, which drops virglrenderer
0001 and brings neshub's codec fix.
A client could connect and state its codecs before the game's encoder
was listening; the command was dropped, the encoder started on AV1, and
a client without AV1 got sound and no picture. Commands now wait for the
encoder's socket, and a frame in a codec no client decodes sends the
agreed set again.
The codec the clients agree on goes to the encoder over a socket that does
not exist until the game draws, and a client can connect first. The command
was dropped, the encoder started on AV1, and a client without AV1 sat on a
splash with sound and no picture. Commands now wait for the socket, and a
frame in a codec no client decodes sends the agreed set again, which also
covers a launcher handing over to a game with an encoder of its own.
xs is a box with no stream, sm 720p, md 1080p, lg 1440p, xl 4K. Each
resolution keeps its price, so md is now 1x. New boxes default to md;
existing boxes move up one size.
xs is a box with no stream, sm 720p, md 1080p, lg 1440p and xl 4K. Each
resolution keeps its price, so md is now 1x. New boxes default to md, and
existing boxes move up one size so none of them drops a resolution.
The size column's migration was listed in the journal but never
committed, so migrating a fresh database failed and the last prod
release stopped before deploying.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
POST /session takes an optional `tier`; the start job, the allowance
check and the burn rate use it, falling back to the box's tier. Adds a
nullable `session.tier` column.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
POST /session takes an optional tier. The host starts the run at that
size, the allowance is checked against it, and the run is billed at it.
A run asked for without one runs at its box's size, as before.
The size belongs to the run rather than the box: a box is a game's
install on a machine, and the size is chosen each time it is played, so
playing bigger never means a second box and a second install.
GET /session returns the caller's runs that have not stopped, so a
client refused with a busy box can rejoin the same game or offer to stop
the other one.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Asking for a run on a busy box is refused, but the refusal does not say
what holds the box. A client that just lost its connection needs to know
whether the run in the way is the game it asked for, which it can rejoin,
or another one it should offer to stop. GET /session answers that.
GET /library picked each game's most recent download row from any host,
so one person's library could show another person's machine id and
progress. It now reads only hosts the caller's boxes are on.
Boxes and library downloads gain `gpu`, the host's first reported card
model, so clients name a place by hardware rather than a machine id.
Null until a host reports one.
Tests: core 240 pass (3 new), api 171 pass. The leak test was checked to
fail with the host filter removed.
GET /library picked each game's most recent download row from any host,
so one person's library could show another person's machine id and
progress. It now reads only hosts the caller's boxes are on.
Boxes and library downloads gain `gpu`, the host's first reported card
model, so a client names a place by hardware a person recognises rather
than by a machine id. Null until a host reports one.
- `POST /session/:id/stop`: the owner ends a run. Adds
`session.time_stop_requested` (migration 0022) and a `user` reason on
`session.stop` jobs.
- The box reports how many clients are watching (`viewers` lifecycle
message), so a host can end a run nobody uses.
- `GET /machine/:id/downloads`: the owner reads what their machine is
downloading and has ready.
GET /machine/:id/downloads lists every game the machine is downloading,
has ready or failed to download, with progress and each game's name.
The library already carries a download per game, but it is the most
recent one across every host, which is not what a person watching their
own machine set up wants to see.
Owner-scoped in the query: a machine that is not yours answers 404, the
same as one that does not exist.
A host cannot tell a run somebody is playing from one nobody has looked
at in an hour, so it cannot end the second without guessing.
The hub now answers the address socket with a second line,
`viewers <n>`, counting clients that are streaming. Init reads it
alongside the address and sends a new `viewers` message to the host
when the count changes. A reader that takes only the first line gets
the address exactly as before, a server that never writes the second
line is still read, and a host that does not know the message skips it,
so either end can ship first.
POST /session/:id/stop. A run no host has claimed ends at once; one that
is starting or live is marked with time_stop_requested and its host is
handed a session.stop with reason `user` on every poll until it reports
the run ended. The host already ends a run the same way whatever the
reason, so no host change is needed.
Asking twice keeps the first time asked, and asking of a finished run
changes nothing. Somebody else's run answers 404, and a host cannot
stop a run as its owner.
- The host report accepts `gpus` (model, VRAM in MiB, PCI id); the
machine keeps them and `/machine` returns them. One new nullable column.
- Must reach prod before host agent 0.2.10, which sends the field.
The host report gains gpus: each card's model, VRAM in MiB when the
driver says, and the PCI id it was named by. Kept on the machine (a jsonb
column, written only when it changes) and returned by /machine. What a
person calls the machine, and the first measured part of the capacity a
scheduler will place by.
- The API accepts a host's own tier label in a box report. Every report
naming a box was rejected with a 400, so stop reasons never reached a
run.
- A library sync now removes the games the account no longer owns. An
empty sync removes nothing, and a game owned again comes back.
- Build: a mesa patch keeping the GE rings off the discardable path.
A sync is the whole of what the account owns, but it only ever upserted,
so a revoked licence or a library from a previous Steam account stayed
listed forever. A sync that fully lands now soft-deletes the rest; an
empty list prunes nothing, and a game owned again comes back.
The report schema pinned each box's tier to our size enum, but hosts name
a box's shape by resolution (1080p). Every report naming a box was a 400,
so stop reasons never reached the run. Placement already stores the size;
the reported label is free text and is not stored.
- **Installer:** piped into `sudo`, installs to `/usr/local/bin`, and
hands onboarding `/var/lib/nestri`. Root lets the host agent jail each
box under its own uid and give guests a network. A host that ran the
agent as the sudo user has that state copied over (never moved), so it
keeps its machine identity and its downloads. Pins host agent 0.2.4.
- **Base image:** `vhost-user-nvgpu` and `nvgpu-userspace`, static
(musl), from the same `NVGPU_REF` as the guest driver and Vulkan layer,
under `/boot/host`. The host agent takes them out at unpack, as it does
the kernel, and forwards the GPU when it finds the NVIDIA driver loaded.
Merge after the host agent 0.2.4 release: the installer pins it.
Re-running the install command on a host that runs the agent failed:
onboarding refuses to race the agent holding the host lock. The
installer now stops `nestri-host.service` first; onboarding starts it
again on the new binary.
A `kernel-build` stage runs `kernel-build.sh` and puts the result at
`/boot/vmlinux`; host onboarding takes it out beside the root image, so
kernel and image (and their shared NVGPU_REF) always ship as one pair.
`make push` publishes `:latest` and a commit tag from the build box.
Built on the build box: `/boot/vmlinux` is 24 MB, and
`vmlinux.nvgpu-rev` is beside it.
A QR sign-in on Nestri's own hardware proves the Steam account, so `POST
/machine/enrolment` from a machine in `FLEET_ORGANISATION_ID` also links
it to the person. Hosts somebody brought never link (a machine vouching
for itself could attach any Steam account to anyone enrolled on it). A
Steam account already linked elsewhere stays there; the report is
recorded either way.
Tests: three new in `enrolment.test.ts` (fleet links and is idempotent,
own host does not link, taken account is left alone). 164 pass.
- A Steam account is linked only through Steam's own OpenID sign-in
(POST /steam/link/start, GET /steam/link/callback). POST /steam/link,
which linked any Steam id on trust, is removed. Migration 0020,
additive.
- The installer installs host agent 0.2.2 (virtiofsd preflight, install
wording), released from nescore prod.
POST /steam/link took a Steam id from the request body and linked it. A
Steam id is public, so anyone could claim any account, its library, and
the one free weekend that is keyed on it.
Linking is now POST /steam/link/start, which returns a Steam sign-in URL
tied to a one-time, ten-minute request, and GET /steam/link/callback,
where the browser comes back. The callback accepts the id only if Steam
confirms the assertion, the assertion was issued for exactly this
callback, and the id is read from Steam's claimed identity URL in the
one shape Steam issues it. The request is spent on use.
One line: the installer's default host agent version is now 0.2.1,
released from prod. It ends a run when the job poll sends session.stop,
which the free weekend relies on.
Ships everything on dev to prod, including:
- POST /box: create a box on your own host or on the fleet
- Fleet entitlement during your own run; fleet runs billed to the box
owner's personal team
- The free weekend trial: Fri–Sun, 2 hours, one per team, email and
Steam account, off once five teams pay; session.stop jobs for overdue
trial runs
- Install tokens that register a host to an organisation's fleet
- Migrations 0018 and 0019, both additive
Plus the sign-in screen work, the guest image changes and the team/box
reads already on dev.
Fleet placement stays off until FLEET_ORGANISATION_ID is set on the API
host.
An install token can now name an organisation instead of a team, and the
host it registers belongs to that organisation's fleet. Only members of
the organisation may mint one, since fleet hardware is rented to other
people. A token for both is refused, by the route and by a check
constraint, as it already is for a machine.
A team with no plan may run on the fleet from Friday 00:00 to Monday
00:00, Helsinki time, for two hours of live play per weekend. One trial
per team, per email address and per Steam account; the claim outlives
the account, so closing it and signing up again is not a second trial.
Once five teams pay, the trial closes for everyone.
A trial run is marked on the session. Its time is counted from the run's
own live interval and kept out of the plan's allowance. Unlike an
allowance, the trial ends a run: the host's work poll now carries a
session.stop job for any trial run past its hours or its window, which
an agent that does not know the kind already skips.
A fleet machine belongs to an organisation, which is not something that
pays, so fleet runs were neither checked nor metered. The payer is now
the box owner's personal team, and the meter that starts and stops with
a run asks the same function the start check does, so the team checked
and the team billed cannot differ.
Fleet machines refused everyone, because nothing could yet say who had
paid for them. A run on one can only exist once it passed the start
check, so a starting or live run of your own box on that machine is now
what grants access, and it ends with the run. Holding a box is not
enough: a box costs nothing to hold.
POST /box places a box once, at creation. On your own host it keeps the
existing rule (exactly one host, or refuse). On the fleet it picks an
online machine owned by the fleet organisation, the one holding the
fewest boxes. Each person holds at most one fleet box. Which organisation
is the fleet is deploy configuration, so no deployment rents hardware it
was not told to.
No machine online is a 429, since the same request succeeds once one is
back.
On an NVIDIA host, NVIDIA's Vulkan driver in the guest lists ray
tracing, optical flow, binary import and CUDA kernel launch, then fails
`vkCreateDevice` with `VK_ERROR_INITIALIZATION_FAILED` for each of them.
An engine that enables what it is offered does not start. virtio-nvgpu's
guest layer (`vklayer/`, nestrilabs/virtio-nvgpu#5) hides those
extensions. It acts only on an NVIDIA device in a guest that has the
virtio-nvgpu driver, so on AMD and Intel boxes it loads and does
nothing.
- A new `nvgpu-layer-build` stage builds the layer from virtio-nvgpu at
`NVGPU_REF`, the same setting the kernel build uses for the guest
driver. The commit is recorded in `/usr/share/nestri/virtio-nvgpu.rev`.
- The layer and its manifest go in `/usr/lib` and
`/usr/share/vulkan/implicit_layer.d`. The missing-library check covers
the layer.
**Merge after nestrilabs/virtio-nvgpu#5.** The default `NVGPU_REF=dev`
has no `vklayer/` until that PR lands, so `make build` fails at this
stage before then.
Built on nestripc-1 with `NVGPU_REF=feat/v0.2-compute` (virtio-nvgpu
112eac4) and pushed:
- `ghcr.io/nestrilabs/nestri/base:latest` sha256:152a4485…
- `ghcr.io/nestrilabs/nesbox/jail:latest` sha256:df813051…, rebuilt on
that base from nesbox dev 6d37597, which includes the vhost-user
forwarding
Measured on box2 (RTX 3060, 595.104.02) with virtio-nvgpu's vkrt probe:
with the layer on, a plain device is created and none of the seven
extensions are offered. With it off, all seven are offered and fail with
-3.
On an NVIDIA host the guest's Vulkan driver offers ray tracing, optical
flow and a few others it cannot create a device with, and an engine that
enables them does not start. virtio-nvgpu's layer hides them; it acts
only on a forwarded NVIDIA device, so AMD and Intel boxes are unaffected.
Built from virtio-nvgpu at NVGPU_REF, the ref the kernel build already
takes the guest driver from, and recorded in the image.
Polishes the sign-in screens and adds a way to see all of them locally.
**Screens**
- Code fields are drawn as one box per character (device code shows a
dash between halves). One real input sits under the boxes, so typing,
paste and autofill work normally, and with script off it falls back to
the plain field.
- A wrong answer or a refused character shakes the field and rings it
red.
- Focus is white instead of the brand colour; the button stays dim until
the form can be sent.
- One alert at a time: an error replaces "Code sent to …" instead of
joining it.
- The device confirm screen says "A device" instead of the client's
self-chosen name, with the Deny warning moved under the buttons.
- A device code typed with its dash (`BCDF-3467`) is no longer cut to
`BCDF-346`.
**Preview**
`bun run preview` in `apps/auth` serves all 18 screen states on :1338,
at desktop and phone sizes, reloading on save. No database or mail
needed.
To make that possible the issuer's screens moved out of the route
handlers into `packages/auth/src/ui/issuer.ts`, unchanged.
Tests: `bun test` in `packages/auth` (67 pass).
A code field silently dropped a character it does not accept, so typing
a letter into the emailed code, which is digits only, looked exactly
like a field that does not work. It now shakes the boxes and rings them
red, the same answer a wrong code gets. Dashes and spaces are still
taken quietly, since they are how a code is written.
The shake an error page plays on load now rings the field red too,
fading back to its usual border once it settles, on every field
including the email address. Reduced motion keeps the red and drops
the movement, since the red is the message.
The boxes were one input each, with script moving focus between them.
That worked in a clean browser and failed in an ordinary one: on the
emailed code, whose first box is marked as a one-time-code field, typing
and pasting did nothing, while the device code, which is not marked,
worked. A password manager or autofill prompt hooking that field and a
script moving focus between six fields were working against each other.
It also left gaps: pasting into the third box with the second one empty
produced a code with a character missing.
Now the real input is stretched invisibly over the boxes and is the only
thing typed into, pasted into or autofilled; the boxes only display what
it holds, with the next one ringed and showing a caret. With nothing
moving focus there is nothing to fight, and a code cannot have a gap.
The browser's own validation message works again too, since the field
it points at is where the boxes are.
The emailed code and the device code are now typed into a row of boxes,
with a dash drawn between the device code's two halves. Typing moves to
the next box, backspace on an empty box goes back, the arrow keys move,
and a pasted or autofilled code, dashes and all, is spread across them.
It is an enhancement of the field and not a replacement for it. The
server still sends the single plain input, which works on its own with
script off; a short inline script draws the boxes over it and keeps it
in step. The real input is still the one that submits and the one whose
validity dims the button, so nothing else had to learn about boxes.
Pressing the button early moves to the first empty box, since the
browser's own message would point at a field nobody can see.
A device code is shown as BCDF-3467 and will be typed or pasted that
way. The field allowed exactly 8 characters, so a correct code was cut
to BCDF-346 before the server, which ignores separators, ever saw it.
A grouped code's field now has room for its separators and a pattern
that allows a dash or a space between groups, and the device code is
declared as two groups of four.
