The host report gains gpus: each card's model, VRAM in MiB when the
driver says, and the PCI id it was named by. Kept on the machine (a jsonb
column, written only when it changes) and returned by /machine. What a
person calls the machine, and the first measured part of the capacity a
scheduler will place by.
A sync is the whole of what the account owns, but it only ever upserted,
so a revoked licence or a library from a previous Steam account stayed
listed forever. A sync that fully lands now soft-deletes the rest; an
empty list prunes nothing, and a game owned again comes back.
The report schema pinned each box's tier to our size enum, but hosts name
a box's shape by resolution (1080p). Every report naming a box was a 400,
so stop reasons never reached the run. Placement already stores the size;
the reported label is free text and is not stored.
- **Installer:** piped into `sudo`, installs to `/usr/local/bin`, and
hands onboarding `/var/lib/nestri`. Root lets the host agent jail each
box under its own uid and give guests a network. A host that ran the
agent as the sudo user has that state copied over (never moved), so it
keeps its machine identity and its downloads. Pins host agent 0.2.4.
- **Base image:** `vhost-user-nvgpu` and `nvgpu-userspace`, static
(musl), from the same `NVGPU_REF` as the guest driver and Vulkan layer,
under `/boot/host`. The host agent takes them out at unpack, as it does
the kernel, and forwards the GPU when it finds the NVIDIA driver loaded.
Merge after the host agent 0.2.4 release: the installer pins it.
A QR sign-in on Nestri's own hardware proves the Steam account, so `POST
/machine/enrolment` from a machine in `FLEET_ORGANISATION_ID` also links
it to the person. Hosts somebody brought never link (a machine vouching
for itself could attach any Steam account to anyone enrolled on it). A
Steam account already linked elsewhere stays there; the report is
recorded either way.
Tests: three new in `enrolment.test.ts` (fleet links and is idempotent,
own host does not link, taken account is left alone). 164 pass.
The guest's audio stream ran about 3.5% short of realtime: 193 packets a
second where 200 were due, with no network loss at all. The client plays
at its own 48 kHz, so its buffer drained and underran every few seconds,
idle or loaded.
The graph was running 128-frame (2.67 ms) cycles. The 256 floor for VMs
matched on `cpu.vm.name`, which PipeWire reads from DMI and nesbox does
not provide, so it never applied, and neswire's 240-frame request was
rounded down to a power of two. A busy guest missed ~20 of those cycles a
second (pw-top ERR on neswire), and each missed cycle's audio never
reached neswire while the graph clock moved on.
- PipeWire is fixed at 240 frames, power-of-two rounding off: one cycle,
one 5 ms Opus packet, twice the headroom of before.
- neswire reads the graph position each cycle and fills exactly the
frames a missed cycle skipped, as silence, so the stream stays at
48 kHz whatever the scheduler does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each client states what it decodes when it connects, and the hub passed
each statement to the encoder as it arrived. The newest client decided, so
a client that only decodes H.264 lost its picture as soon as one that
decodes AV1 or H.265 joined.
The hub now keeps every client's statement and gives the encoder their
intersection, recomputed when a client joins, when one leaves, and when the
controller returns to automatic. A room settles on the best codec all of
its clients share, and moves up again once the client holding it back has
left. Manual mode still leaves the encoder alone; the change is applied
when the mode goes back to automatic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /steam/link took a Steam id from the request body and linked it. A
Steam id is public, so anyone could claim any account, its library, and
the one free weekend that is keyed on it.
Linking is now POST /steam/link/start, which returns a Steam sign-in URL
tied to a one-time, ten-minute request, and GET /steam/link/callback,
where the browser comes back. The callback accepts the id only if Steam
confirms the assertion, the assertion was issued for exactly this
callback, and the id is read from Steam's claimed identity URL in the
one shape Steam issues it. The request is spent on use.
An install token can now name an organisation instead of a team, and the
host it registers belongs to that organisation's fleet. Only members of
the organisation may mint one, since fleet hardware is rented to other
people. A token for both is refused, by the route and by a check
constraint, as it already is for a machine.
A team with no plan may run on the fleet from Friday 00:00 to Monday
00:00, Helsinki time, for two hours of live play per weekend. One trial
per team, per email address and per Steam account; the claim outlives
the account, so closing it and signing up again is not a second trial.
Once five teams pay, the trial closes for everyone.
A trial run is marked on the session. Its time is counted from the run's
own live interval and kept out of the plan's allowance. Unlike an
allowance, the trial ends a run: the host's work poll now carries a
session.stop job for any trial run past its hours or its window, which
an agent that does not know the kind already skips.
A fleet machine belongs to an organisation, which is not something that
pays, so fleet runs were neither checked nor metered. The payer is now
the box owner's personal team, and the meter that starts and stops with
a run asks the same function the start check does, so the team checked
and the team billed cannot differ.
Fleet machines refused everyone, because nothing could yet say who had
paid for them. A run on one can only exist once it passed the start
check, so a starting or live run of your own box on that machine is now
what grants access, and it ends with the run. Holding a box is not
enough: a box costs nothing to hold.
POST /box places a box once, at creation. On your own host it keeps the
existing rule (exactly one host, or refuse). On the fleet it picks an
online machine owned by the fleet organisation, the one holding the
fewest boxes. Each person holds at most one fleet box. Which organisation
is the fleet is deploy configuration, so no deployment rents hardware it
was not told to.
No machine online is a 429, since the same request succeeds once one is
back.
send_many_datagrams queues a frame's fragments under one lock hold and
one driver wake instead of one per fragment. It also rejects the whole
frame on TooLarge, so a path that shrinks mid-frame no longer leaves half
a frame on the wire for the client to time out. Eviction behaviour is
unchanged. A refused PyroWave frame now keeps its sequence number, since
nothing went out under it. pyro_bench mirrors both ends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The host can now ask a box for a shell on a pseudo-terminal, the way a container
runtime attaches to a container. `HostToGuest::Attach` starts the process, and
the guest then dials the host on a vsock port of its own (7001) and carries the
terminal's bytes over a small framed stream: data both ways, a resize, a hangup,
and the exit status last. A screenful of output on the JSON-lines control channel
would hold every other message in the box behind it, which is why it is not
there. A failure to start is the exception, and is said on the control channel
because nothing has connected to the data port yet.
A shell is not a launch: it never ends the box, and the box ending ends it. It
is started through the same reaper as everything else, so it cannot become the
zombie that exists to be prevented. With a terminal on the other end the PTY is
ordinary; without one it is put in raw mode so a command run for its output gets
its bytes back unrewritten. Closing the stream hangs the shell up, process group
and all.
The frame codec is a pure bytes-in, frames-out module in nesprotocol with a
payload cap and no skipping of unknown tags, since a length read from a guest is
read across a trust boundary. No version bump: adding a message does not need
one, and a guest that predates it simply never dials.
The runtime stage gains strace, lsof and less, so a root shell in a box is not
one that cannot trace a process.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There is no service manager in a box, so nothing applies the distribution's
sysctl configuration, and the guest ran on the kernel's own defaults. Two of
them are too low for a modern engine under Proton:
- vm.max_map_count defaults to 65530. A large Unreal Engine 5 title maps far
more regions than that, and the failure is a null allocation deep inside the
engine or Wine rather than a message about mappings. It is set to 2147483642,
the value SteamOS and the distributions that package Proton use. It is read
back after the write, because an accepted write that did not take reports
success while the box is still unconfigured.
- RLIMIT_NOFILE is raised on init, which every process in the box inherits. A
child can lower a limit but cannot raise its hard one, so the unprivileged
workload could never get above what it started with. 524288 is the hard limit
Proton's documentation asks for. It is only ever raised.
Both are best effort and each failure says what it costs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The confirm screen named the client asking to sign in. That name is
chosen by whoever started the request, so it let a stranger's request
introduce itself as anything at all; and to the person reading, it is
just a device. It now says "A device is asking to sign in", and a test
checks that a client's self-chosen name never appears.
The warning about choosing Deny moves under the two buttons, as small
print in the same style as the terms line on the email screen, so the
question and its two answers read first.
bun --watch watches each imported file, and many editors save by
writing a new file and renaming it over the old one. The watch does not
follow the rename, so the first such save reloaded the preview and every
later one was silently ignored.
The preview now runs under a small supervisor that watches the source
directories and restarts the server, which sees a save however it was
made. It also stops the server when it is stopped itself, so the port
is not left held.
bun run preview (in apps/auth) serves a gallery of all 18 states the
sign-in flow can show, at desktop and phone sizes, each one also
viewable full size. It needs no database or mail: each state is the
real screen value, drawn by the real renderer with the real theme.
bun --watch restarts the server on any change to the renderer, the
stylesheet or the copy, and every page polls for the restart and
reloads itself, so styling is edit, save, look.
The issuer's screens were written inline in its route handlers, so the
only way to look at one was to reach the condition that produces it: an
expired cookie, a refused grant, ten wrong guesses. They now live in
ui/issuer.ts and the handlers call them, with no change to copy, status
or markup.
The app's theme moves to its own module for the same reason: it can be
used without importing the issuer and its database stores.
The boot descriptor gains `images`: read-only EROFS images mounted on their
own, with no writable layer over them. nesinit mounts them first, before
overlays and shares, with the same nosuid/nodev guards every layer has and
without noexec because what is in one is run. CONTROL_VERSION moves to 5: the
descriptor refuses fields it does not know, and a guest image without Proton is
useless to a host that does not send one, so the mismatch is named at the
handshake.
The guest rootfs no longer contains Proton. The checks that used to guard the
in-image copy (every library its programs and Wine's core link, an interpreter
behind the entry point) move to a compat_check stage that verifies the Proton
named by PROTON_IMAGE and leaves a marker in the image saying which one. The
leaf stages copy that marker out of it so the stage cannot be skipped as
unused. With nothing at a Proton path, a box started without one fails saying
where it looked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
At this point, why not..
Calling it `nespyro` since it's slightly modified, uses slang shader
language instead, along other things..
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
POST /team makes another team with the caller as owner. The website now
serves a team at nestri.io/<slug>, so Team.Slug refuses its top-level
pages (home, new, pricing, ...) and a personal team whose generated slug
would be one gets a suffix instead.
GET /team, PATCH /team/:id (owner or admin; a taken slug is a 409 on
slug), GET /team/:id/members, GET /box. A team you are not on is a 404
everywhere, so ids cannot be probed. Team.Name and Team.Slug are exported
so a client checks the same rule the API does.
GET /install.sh serves a POSIX script, embedded in the API at build time
so the script and the routes that redeem its token ship together. It
checks the platform, asks where box images should live, downloads the
host agent at a pinned version, verifies it against SHA256SUMS, installs
it for the calling user and hands over with the token in the environment
rather than argv.
GET /install/:component/:version/:asset redirects to a one-minute signed
URL on a private S3-compatible bucket, so every download passes through a
route that can be logged or switched off. The SigV4 signer is written
against Web Crypto and checked against AWS's published example.
POST /machine/install-token mints a token for a team the caller belongs
to, with the same team resolution and membership rule as /register.
POST /machine/install needs no session: it spends the token and returns
the same id, slug and one-time secret as registering directly, refusing
unknown, expired and used tokens identically.
Adds `nesgamepad`, basically "diet-coke vimputti" for Nestri's microVM
needs here.
Comes with direct mapping for:
- Sony
- DualShock 4 (v2), DualSense
- Microsoft
- Xbox 360 pad, Xbox One S pad
- Nintendo
- Pro Controller
- Generic pad for unknown controllers
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
"No suitable memory type for DMA-BUF import" names neither mask, so it cannot
tell a driver that refused the descriptor outright (fd_type_bits=0) from one
whose memory types simply do not overlap the image's — which are different
bugs in different components. The three numbers were already computed for a
debug! two lines above, and a debug line only fires at a level nobody runs in
the field; this one did not print at all while a neighbouring crate's did.
Now the failure carries image_type_bits, fd_type_bits, their intersection, and
the size, format, modifier and stride it was importing. On a virtualised NVIDIA
node this turned three unknowns into one measurement and pointed straight at
the kernel side.
The comment explaining the `cmd /C start` bug spelled out the cut submit
link in full. This repository is public and is pull-mirrored onto forges
that crawlers index far more eagerly than GitHub does, so GPTBot found it
there and followed it -- twice. The worker classed both as truncated
submissions, correctly, and raised an alert apiece for a bug fixed in
0.2.1.
Describe the cut URL instead of writing it, and say why, so the next
person documenting a bad link does not paste a working one.
Three comments named a file in the internal repo: a contract, a design
guide, and a user-research document with section numbers. A filename plus
a section tells a reader what to ask for, which is the escape the marker
convention exists to prevent.
Each comment now states the requirement on its own terms. The render-node
requirement carries a decision marker; the other two have no decision
behind them, so they cite nothing.
Nine commits, in the order they are easiest to read.
## The shared operator secret is gone
`x-nestri-admin-token` had no caller left — the device pairing it
existed for is
on hold, and nothing in any tree sent it. What remained was a key that
bypassed
authentication entirely and was required to boot.
Every route behind it had a better answer. The two Steam sync routes
took a
`userId` **in the body**, so one secret could write into anybody's
library; they
now authenticate as the host holding that person's Steam sign-in, and
the claim
is checked against the enrolment record. Download-state narrows to hosts
alone.
Creating a game by hand is deleted (syncing already upserts the
catalogue), as is
reading the waitlist — every address on it belongs to someone who has
not agreed
to anything, and answering it over HTTP made that list something a
leaked key
could drain.
Nothing in the API now accepts a credential standing for more than one
caller.
## An organisation owns hardware
Two kinds of machine were modelled as one. A host somebody brings is
theirs,
reached through a team, and should die with their account. A host bought
to serve
other people's workloads is none of those things, and had to be
registered under
an employee's personal team — where their account going away took it
with them.
Ownership is now an either/or, enforced by a check constraint rather
than a
convention: both null is a host nothing can bill, and both set is two
answers to
"whose is this?". Membership of an organisation is derived from a
verified email
domain rather than stored, so signing in with a personal address still
gets an
ordinary personal account.
Not a billing subject. A team pays for what it uses either way.
## Burn, and the three windows
The unit is one second of a reference session, so an allowance is
measured in
time and a bar prints the stored number rather than converting into it.
Counters are stored beside the time they began, and a total whose stamp
has
rolled outside its window reads as zero — so a window clears without
anything
running. No schedule to misfire, no race between a reset and a write.
Two rules on the allowances are enforced rather than remembered: an
allowance
must exceed its own window, or one uninterrupted session hits a wall;
and each
longer one must sit under what the shorter already permits, or it never
binds and
is decoration.
Burn is recorded as segments at one rate, because a run's rate does not
survive
its own lifetime. On our own hardware a bigger tier costs more; on a
caller's own
card it does not, since there is no share of a card of ours being spent.
The gate is at the one moment it may speak — before a run starts, never
again. A
limit refuses the next run and never interrupts one going.
## Payment
Checkout, portal, and a webhook. No price, no currency and no card
detail is
stored: a subscription's existence and its state are the whole of what
crosses
back. Free is a real subscription too, created outright since nothing a
month
needs no payment, so an upgrade changes a subscription rather than
inventing a
customer.
The webhook is the only route no session protects. A signature over the
raw body
stands in for one, checked before the body is parsed, and with no secret
configured it refuses everything.
## Breaking
- `x-nestri-admin-token` is no longer accepted; `ADMIN_SHARED_SECRET` is
no longer read
- `POST /games`, `GET /waitlist` and the `/pairing-code` routes are gone
- `POST /games/sync` and `POST /library/sync` now need host credentials
and take `userId` in the body
- `POST /steam/link` no longer accepts `userId`; `POST
/games/download-state` no longer accepts `hostId`
- `POST /session` responds `{ data, billing }` rather than `{ data }`
## Checks
342 tests pass. Typecheck unchanged from before the branch — the two
pre-existing
errors in `utils/hook.ts` and `utils/validator.ts` are untouched.
Billing is inert until configured: unset `POLAR_*` means every team is
free, no
checkout starts, and the webhook refuses every delivery.
Checkout, the customer portal, and the webhook that moves a team's plan.
Nothing money-shaped is stored. No price, no currency, no card detail — a
subscription's existence and its state are the whole of what crosses back,
because they are the only two facts the product needs and anything more would
be a second copy of a record somebody else is authoritative for.
Currency is deliberately not ours to hold. A product carries a price per
currency on their side and the customer's location picks one at checkout, so
there is no figure in this API that could drift from the one somebody is
charged. The team id travels as the customer's external id, which keeps the
mapping on their side rather than putting a foreign primary key in our schema.
Access follows their state, and the interesting cases are where that is not the
same as "paying right now". Cancelling keeps the plan: they paid to the end of
the period and turning them off when they click it takes something they bought.
A failed card keeps it too, because a retry that ends in payment should not
have cost them access in the middle. Only a revoked subscription takes it away,
which is the one moment nothing is left that was paid for. An event we do not
recognise changes nothing at all — new types are added by people who do not
know what we do with them, and a default that moved a plan would eventually
cancel an account nobody cancelled.
The webhook is the only route here no session protects, because its caller has
no account and never will. A signature over the raw body stands in for one, and
it is checked before the body is looked at — a body that has been parsed and
re-serialized is not the body that was signed. With no secret configured it
refuses everything rather than accepting anything, since otherwise knowing the
URL would be enough to set somebody's plan. Note also what is absent: no route
sets a plan, so there is no endpoint for granting yourself a subscription.
The product is written down as a definition with a script rather than clicked
into a dashboard, because the two environments are separate servers and nothing
made in one can be moved to the other. Promoting it is running the same script
with the other token, which is the only version of that which cannot drift. It
writes nothing without --apply and refuses to add a second product with a name
already taken.
The rate was a constant. That was correct for everything the system can
currently run and wrong the moment it can run anything else, because a tier
buys a share of a card — so a bigger one on our own hardware is more of
something we bought being spent, and a flat rate there sells a whole card for
the price of a quarter of one.
So a run's rate now comes from what the run is: its size tier, and whose
hardware it sits on.
On the caller's own hardware the tier changes nothing. There is no share of a
card of ours in play, so a run costs one unit a second whatever size it asked
for. Charging somebody more for taking more of a GPU they bought is a tax on
their own hardware, and not doing that is most of what this model is for.
This exposed a bug in what went before. Resegmenting recomputed one shared rate
and wrote it to every open stretch, which was harmless while all runs cost the
same and would have quietly repriced an expensive run as whatever the last one
to start was. Each stretch now keeps its own rate, which is also the more
honest shape: a run's rate is a property of that run, and nothing about it
changed because a sibling appeared or the clock ticked.
The account's total is now the sum of what its runs cost rather than a count
times one rate — an expensive run and a cheap one alongside it are not two of
anything. Concurrency still lands exactly where it did, as there being more to
add, and no run gets dearer because another started.
There is no hardware factor yet and its absence is deliberate: nothing records
which card a host has, so a table keyed on a model would be keyed on nothing.
A faster card should cost more, and that starts with a column.
The reference tier is pinned at exactly one unit a second, checked rather than
assumed. The unit is a second of a reference session, so moving it would
silently redefine every allowance — the same stored number would mean a
different number of hours.
Wanjohi
DatCaptainHorseandClaude Opus 5.5
