Step 1 of nescore's 2026-10-16 media timing spec. Media carried no
send time and audio no time at all, so every layer had to infer lateness
from gaps between arrivals.
- clock: one sender clock, CLOCK_MONOTONIC in u64 microseconds, shared
by every process that stamps media. Off unix the client uses its own
process clock, which is all a receiver needs.
- Datagram headers (and PyroWave's) carry sent_us, stamped by neshub at
hand-off to QUIC. Keyframe streams open with it too.
- Video carries captured_us (from nescapture's present time) in place of
milliseconds since the encoder started. Audio carries its first
sample's position on the PipeWire graph clock, which neswire already
read to fill missed cycles. The IPC stamp is a u64 carrying either.
- delay: microseconds. The baseline no longer expires on a 30 s timer:
clock skew is measured as the least-squares slope of each second's
minimum one-way delay, and only a path change resets the baseline.
observe() returns each unit's queueing delay.
- ReceiverReport carries video and audio queueing delay summaries for
the rate control that will consume them.
- Shared encode/decode helpers for video and audio payloads, so the hub
and receivers stop hand-building byte offsets.
No compatibility kept. STREAM_VERSION is 4. nescore needs a pin and the
client side (step 2) before it can talk to this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With RLIMIT_RTPRIO alone, neswire's client loop went SCHED_FIFO 83 and
the PipeWire server's graph loop stayed SCHED_OTHER. The server's
module-rt asks for nice -11 first, and when that is refused (Max nice
priority 0) it hands off to rtkit, which grants nothing in a box. The
client config asks for no nice level, which is why only neswire got it.
RLIMIT_NICE 39 (nice down to -19) next to rtprio 95, as PipeWire's own
25-pw-rlimits.conf grants.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Inside a box, every PipeWire, wireplumber and neswire thread was running
SCHED_OTHER. The rtkit -> polkit -> system bus chain started but granted
nothing. With a game keeping the vCPUs busy, the graph's 5 ms cycles ran
late. pw-top counted no xruns, but neswire sent packets in bursts, and
the client saw 15-25 ms gaps in audio alone (video arriving on time in
the same seconds), which emptied its 10 ms cushion every 30 s or so.
pipewire and neswire now exec with RLIMIT_RTPRIO 95, set before the uid
drop. module-rt checks that first and sets SCHED_FIFO itself, which is
how PipeWire is meant to run outside a desktop session. The workload
doesn't get the grant, so a game can't take the CPU away from the graph.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
neslet now puts nestri.port= on the boot line and installs its forward
and pinned masquerade for that port before the box boots. nesinit hands
it to neshub as NESTRI_BIND_PORT, and neshub binds 0.0.0.0:<port> in
place of iroh's random default. Every flow, the relay's included, then
leaves on one public port, and the address the relay reports is the one
peers can punch through to. No nestri.port= keeps the old behaviour.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- **Sessions (#412):** `end_reason` and `time_claimed`, with migration
`0026_session_end_reason` (two nullable columns, readable by the release
it replaces). `POST /session/:id/state` takes an optional `endReason`,
and a run stopped before it went live can now go from `starting` to
`ended`.
- **Installer (#411):**
- `bash <(curl -fsSL https://api.nestri.io/install.sh) -u` updates a
host; `-h` prints help; a plain token installs.
- When the script is not root, it asks sudo itself.
- The installed host agent version is unchanged (0.2.18).
**Ships before host agent 0.2.19**, which sends `endReason` and the new
transition.
Cut from prod with dev's files, so squash histories cannot conflict; the
tree is identical to dev.
```
bash <(curl -fsSL https://api.nestri.io/install.sh) -u # update this host
bash <(curl -fsSL https://api.nestri.io/install.sh) <install-token> # make this machine a host
bash <(curl -fsSL https://api.nestri.io/install.sh) -h # help
```
Piping still works: `curl -fsSL … | sh -s -- -u`.
**`-u` / `--update`.** No token and no questions. On a registered
machine it downloads and checksum-verifies the current host agent, swaps
it in and restarts the service. The agent then pulls whatever its
release changed (a raised image layout re-pulls the guest, jail and
Proton). It refuses:
- on a machine that was never installed, before downloading anything;
- while a box is running, since restarting the agent would end that
session;
- on an unknown option.
On a machine that already has the target version, it says so and exits
0.
**Root, asked once.** The host agent runs as root, so when the script is
started as anyone else it asks sudo once, before it changes anything.
Neither a pipe nor `bash <(…)` leaves a file sudo could run again, so it
fetches the same script to a private temp file, runs that as root with
the same mode and token, and removes the copy as soon as root has it
open. Only the settings somebody actually set (`NESTRI_HOST_VERSION`,
`NESTRI_BIN_DIR`, `NESTRI_BOX_STORE`) are carried across.
**Tested.**
- **On an NVIDIA host:** `--update` took it 0.2.17 → 0.2.18, verified
and restarted. A second run said "already 0.2.18. Nothing to do".
- **On the fleet host, run as an ordinary user** with `bash <(curl …)
-u`:
- it asked for sudo, the root copy ran and said "Nothing to do", and no
temp copy was left behind;
- `-h` prints usage, and an unknown flag is refused.
- **In a clean container:** `-u` refuses with "not a Nestri host yet",
and a run with no token asks for one.
Until now every finished run said only `ended` or `failed`. A user stop,
a trial window running out, an idle timeout, a host restart and a game
quitting were indistinguishable afterwards.
**Schema.** Migration `0026_session_end_reason` adds two nullable
columns to `session`:
- `end_reason`: one of `user`, `window`, `hours`, `idle`, `left_queue`,
`game_exited`, `launch_failed`, `host_stopped`, `host_restart`.
- `time_claimed`: when a host took the run, set on `requested →
starting`.
**API.**
- `POST /session/:id/state` takes an optional `endReason`. With `ended`
or `failed` it is stored. With `starting` or `live` it is a 400 and the
run does not move.
- Leaving it out of a terminal report stores null, so host agents that
predate this keep working.
- `endReason` and `timeClaimed` are returned on every session response.
- The API writes the reasons it decides itself: `user` when a run nobody
has claimed is stopped, and `left_queue` when a waiting run is abandoned
(the `Left the queue` message is kept).
**New transition, `starting → ended`.** A run stopped while its game is
still downloading has failed at nothing, and never reached `live`, which
is where billing starts. Until now the only way out of `starting` was
`failed`.
**Release order: this goes first.** The host agent that sends
`endReason` is a separate change. This route's body schema is strict, so
without this deployed the current API answers that agent with a 400.
**Tests.**
- API: 185 pass, 5 of them new.
- Core: 41 session tests pass, 3 of them new.
- The new tests fail when the implementation is broken on purpose.
- No new type errors.
- **Guest image (#405):** carries `libvpx`, `speex`, `libtheora`,
`libvdpau` and `jsoncpp`, which Proton's GStreamer plugins link. Without
them, `libgstlibav` and `libgstspeex` failed to load in every box on
every Proton shipped so far. Verified: every library both published
Protons need is now provided.
- **Billing (#406):** every Polar webhook logs its type, the team it
resolved to, the plan and status, whether it changed anything, and the
subscription and customer ids. Logging only.
Cut from prod rather than opened from dev: after squash merges the two
histories conflict, though the files differ only in these four. This
branch's tree is identical to dev's.
After merge, the guest image is published from this prod commit. Host
agent 0.2.18 then makes hosts pull it.
**Problem.** A billing webhook that changes nothing answers `200`
exactly like one that worked.
- A subscription made before checkouts carried the team id (`db43068`)
never matches a team.
- When one was cancelled, the API logged `a paid subscription matched no
team` with only the event type. Nothing said whose subscription it was,
so nobody could act on it.
- Deliveries that did apply were not logged at all.
**Changes.**
- `Polar.receive` returns the provider's `subscriptionId` and
`customerId`, in both camelCase and snake_case, as the rest of the
payload is read.
- For a `subscription.*` event, the event's own id is the
subscription's.
- For anything else, such as `order.refunded`, it is the
`subscription_id` field.
- The webhook route logs one line per delivery: `billing webhook {type,
team, standing, applied, subscription, customer}`.
- The unmatched-paid warning carries those ids and says what to do: find
the customer in the provider dashboard and set the team's plan by hand.
- A delivery naming a team that no longer exists is logged as an error,
not silently applied to nothing.
No behaviour change: the same deliveries apply to the same teams. This
is logging only.
**Tests.**
- `bun test src/billing/polar.test.ts`: 17 pass. One new test covers the
ids in both spellings, on a subscription event and on a non-subscription
one.
- `tsc` has no errors in the changed files. The existing errors are in
`utils/hook.ts` and `utils/validator.ts`, from Zod 4.
**Problem.** Proton's GStreamer plugins link `libvpx`, `speex`,
`libtheora`, `libvdpau` and `jsoncpp`, and neither Proton nor the guest
image carries them. Boxes log:
```
Failed to load plugin '.../gstreamer-1.0/libgstlibav.so': libvpx.so.12: cannot open shared object file
Failed to load plugin '.../gstreamer-1.0/libgstspeex.so': libspeex.so.1: cannot open shared object file
```
Wine's media stack then has no decoders, so a game whose videos go
through it plays them as nothing.
**Scope.**
- Not new. The stock `11.0-20261005` Proton and the patched `-n1` build
need exactly the same set.
- The fleet logged it on 2026-10-10 and again on 2026-10-15.
**Fix.** These five packages are added to the guest's runtime package
list.
- They were found by listing every `NEEDED` library of each published
Proton against this image; they were the whole gap.
- They track the sonames Proton was built against, as the new comment
says.
**Verification.** In progress: the image is building, and the library
check is rerun against it for both Protons.
New installs get host agent 0.2.17. It is published at
/install/host/0.2.17/; the binary is checksum-verified and reports
`neslet 0.2.17`. 0.2.17 raises LAYOUT to 6, so hosts rebuild their guest
from the current image (OpenGL frontends, nescope answering every
frame).
Dev still said host agent 0.2.13 and desktop 0.1.0, while prod serves
0.2.16 and 0.1.2. The next dev→prod merge would have sent new installs
back three host releases. This brings dev to prod's numbers. No other
change: the rest of this branch is already on dev.
Guest image fixes from #401, plus the dev changes that had not reached
prod yet.
- **OpenGL games render** (Dome Keeper, confirmed on an AMD host). The
guest's Mesa gains its EGL/GLX frontends, and `vm.overcommit_memory=1`
stops a large reservation being refused before it is used.
- **Chromium / NW.js games get past their first frame** (Sheepy).
nescope answers every commit, and paces Xwayland windows too.
- **Proton patch step.** `build/patches/proton/` and the `-nN` tag.
Nothing uses it until hosts are pointed at the new tag.
- Also included: the pixelforge update already on dev. It is in the
image tested today.
**Versions kept, not downgraded.** Dev still had host agent 0.2.13 and
desktop 0.1.0. The last commit restores prod's 0.2.16 and 0.1.2, so this
merge changes no install versions.
**After merge.** `make push` from this prod commit publishes the guest
image. The host agent release that raises LAYOUT to 6 makes hosts pull
it.
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Games that showed a black screen with sound now render. Found by testing
on an AMD host: none of these were GPU-vendor specific.
**OpenGL games (Dome Keeper, Godot): fixed and confirmed.**
- `c017462`: the guest's Mesa had no EGL/GLX frontends, so a GL game had
nothing to load and never created a device. The image now ships them,
with glvnd's vendor file, so GL reaches zink and the capture layer sees
the frames.
- `5f2ccdf`: `vm.overcommit_memory=1`. The kernel default refused Dome
Keeper's 9 GiB reservation on an 8 GiB box before a byte was used.
**Chromium / NW.js games (Sheepy) stalled after one frame: fixed.**
- `a73c0c4`: nescope answers every commit with frame callbacks and
feedback, not only commits from a surface it maps to a window.
Chromium's GPU process draws for a window another process owns. Wine's
Wayland driver gives that surface no role, so its FIFO present waited
forever in the driver.
- `23ba9b5`: a window is matched by its surface, not its xdg toplevel,
so Xwayland windows are paced too.
**Proton could not create non-ASCII file names: patch added, build
pending.**
- `6770684`: Steam starts games with `LC_ALL=C`, and Wine reads that as
ASCII for every Unix file name. NW.js stops unpacking its app at
`media/• drum loop….webm` and shows an empty window.
- The patch reads an ASCII locale as UTF-8. Proton now takes patches
from `build/patches/proton/<submodule>/`, and a patched image is tagged
`-nN` (`11.0-20261005-n1`).
- Nothing points at that tag yet; hosts move to it in a separate change,
after it is built and tested.
Not covered: Metro 2033 Redux's report looks like the frame-callback
stall, but it was not retested.
Steam starts games with LC_ALL=C, and Wine reads the C locale as ASCII
for every Unix file name it makes, so a name like '• drum loop.webm'
could not be created at all. Sheepy, an NW.js game, unpacks its app from
a zip at start, stopped at that file, and showed NW.js's empty window.
Proton is now patched before it builds (patches/proton/<submodule>/), and
the first patch reads an ASCII locale as UTF-8. The image name gains -nN
for our patch level, so a patched build never takes the plain tag's name.
A Vulkan client presenting FIFO waits in the driver for its frame
callback. Chromium's GPU process draws for a window another process
owns, which Wine's Wayland driver gives a surface with no role, so no
window here matched it and the game presented one frame and waited
forever: Sheepy's black screen. Nothing here paces against a display,
so every commit now gets its frame callbacks and feedback.
Since frame callbacks moved onto the commit path, the commit handler
found a window by its xdg toplevel. An Xwayland window has none, so no
Proton game got a frame callback or presentation feedback. Xwayland then
completes a flip only on its one-second fallback timer, and a game
presenting FIFO stalls in the driver after its first frame.
The kernel's default overcommit policy refuses any single allocation
larger than RAM plus swap. A box has no swap, so the ceiling is the
box's own size, and an engine or driver that reserves address space it
never touches is refused before it uses any of it. Dome Keeper asked
for 9 GiB on an 8 GiB box and never started.
Set vm.overcommit_memory=1 alongside vm.max_map_count, so a request is
granted and the OOM killer acts on what is actually used. In a box that
is the game: init has already exempted itself.
Mesa was built with zink as its only Gallium driver and every frontend disabled, so a game rendering with OpenGL found no GL and showed a black screen. EGL, GLX, GBM and GLES2 are enabled, with the x11 platform for Xwayland clients.
The real change. #399 merged by mistake without it: a rewritten copy of
the #398 merge commit, same tree, so nothing in prod changed. New hosts
install 0.2.16 of the host agent: onboarding retries a failed phase, and
finds steamclient.dll where Valve's client package now keeps it.
A customer the payment provider already knew under the same email kept
their old customer record, without our team id, so their paid
subscription was acknowledged and applied to no team. Checkout now
carries the team in its metadata and the webhook falls back to it; a
paid delivery that matches no team is logged.
The provider reuses a customer it already has under the same email and
leaves our team id off it, so the subscription arrived with no team,
was acknowledged, and changed nothing: a paid plan never applied. The
checkout now carries the team in its metadata, which the subscription
keeps, and the webhook reads it there when the customer has none. A
paid delivery that still matches no team is logged instead of dropped
in silence.
(cherry picked from commit 1301ff0806)
The provider reuses a customer it already has under the same email and
leaves our team id off it, so the subscription arrived with no team,
was acknowledged, and changed nothing: a paid plan never applied. The
checkout now carries the team in its metadata, which the subscription
keeps, and the webhook reads it there when the customer has none. A
paid delivery that still matches no team is logged instead of dropped
in silence.
New hosts install host agent 0.2.15. It refuses NVIDIA's proprietary
kernel module at onboarding, since GPU forwarding needs the open kernel
modules (Turing or newer).
- A Steam sign-in on a person's own host now links the Steam account, as
it already did on ours. Before this, their first launch asked them to
link Steam with nothing in the app to do it from.
- Downloads hand out desktop app 0.1.1: a launch on a person's own host
makes its box instead of stopping at "You do not have a box yet".
- New hosts install host agent 0.2.14, which checks for mkfs.erofs
before it starts setting up.
A sign-in on a person's own host recorded the enrolment but never
linked the Steam account, so their first launch asked them to link it
again with nothing in the app to do it from. The QR sign-in is the same
proof on their host as on ours.
(cherry picked from commit 24ad9f6b78)
A sign-in on a person's own host recorded the enrolment but never
linked the Steam account, so their first launch asked them to link it
again with nothing in the app to do it from. The QR sign-in is the same
proof on their host as on ours.
`/download/windows` and `/download/linux` redirect to a one-minute
signed URL for the current desktop app (`desktop/<version>/` in the
releases bucket). `/desktop.sh` installs it on Linux for the calling
user, without sudo. The version handed out is a constant bumped by a
commit after each release. Tests: 180 pass.
/download/windows and /download/linux redirect to a one-minute signed URL
for the current desktop app, from the same private bucket as the host
agent, so a download page can link to them once and every download still
passes through the API. Which version they hand out is a constant bumped
by a commit after a release is published, as the host installer's is.
/desktop.sh installs it on Linux for the calling user: one binary in
~/.local/bin and a launcher entry, no root.
/download/windows and /download/linux redirect to a one-minute signed URL
for the current desktop app, from the same private bucket as the host
agent, so a download page can link to them once and every download still
passes through the API. Which version they hand out is a constant bumped
by a commit after a release is published, as the host installer's is.
/desktop.sh installs it on Linux for the calling user: one binary in
~/.local/bin and a launcher entry, no root.
Two changes from dev:
- **Runs wait in line for a free slot on their host.** A host says on
its heartbeat how many runs it holds and is handed only that many; the
rest wait, paid before free-weekend, then first come first served.
Reading a waiting run returns its place in line. A waiting run nobody
has asked after for a minute leaves the line. A host that does not
report slots is handed every run, as before, so this is safe to deploy
ahead of the host agent.
- **Free days are Saturday, Sunday and Monday.**
Adds migration `0025_run_queue` (two nullable columns). API tests: 177
pass.
A host holds only so many runs at once, and until now every waiting run
was handed to it regardless; the ones it could not hold failed there.
The host now says on its heartbeat how many runs it holds, and is handed
only as many waiting runs as it has free. The rest wait in line: paid
runs before free-weekend runs, then first come first served. Reading a
waiting run returns its place in line, and says when a paid run that
asked later has gone ahead, so a free player is never reordered quietly.
A waiting run nobody has asked after for a minute was abandoned, and
leaves the line rather than holding up everyone behind it. A host that
does not report its slots is handed every run, as before.
A host holds only so many runs at once, and until now every waiting run
was handed to it regardless; the ones it could not hold failed there.
The host now says on its heartbeat how many runs it holds, and is handed
only as many waiting runs as it has free. The rest wait in line: paid
runs before free-weekend runs, then first come first served. Reading a
waiting run returns its place in line, and says when a paid run that
asked later has gone ahead, so a free player is never reordered quietly.
A waiting run nobody has asked after for a minute was abandoned, and
leaves the line rather than holding up everyone behind it. A host that
does not report its slots is handed every run, as before.
Cherry-pick of aa824a1 from dev, alone.
`GET /box` gains `host` (the machine's DNS label) and `fleet` on listed
boxes. The desktop app uses them to open the fleet machine's Steam
sign-in during onboarding. Additive; nothing existing changes shape.
Tests: 172/172 against Postgres.
DatCaptainHorseandClaude Opus 5.5
Wanjohi