On an NVIDIA host, NVIDIA's Vulkan driver in the guest lists ray
tracing, optical flow, binary import and CUDA kernel launch, then fails
`vkCreateDevice` with `VK_ERROR_INITIALIZATION_FAILED` for each of them.
An engine that enables what it is offered does not start. virtio-nvgpu's
guest layer (`vklayer/`, nestrilabs/virtio-nvgpu#5) hides those
extensions. It acts only on an NVIDIA device in a guest that has the
virtio-nvgpu driver, so on AMD and Intel boxes it loads and does
nothing.
- A new `nvgpu-layer-build` stage builds the layer from virtio-nvgpu at
`NVGPU_REF`, the same setting the kernel build uses for the guest
driver. The commit is recorded in `/usr/share/nestri/virtio-nvgpu.rev`.
- The layer and its manifest go in `/usr/lib` and
`/usr/share/vulkan/implicit_layer.d`. The missing-library check covers
the layer.
**Merge after nestrilabs/virtio-nvgpu#5.** The default `NVGPU_REF=dev`
has no `vklayer/` until that PR lands, so `make build` fails at this
stage before then.
Built on nestripc-1 with `NVGPU_REF=feat/v0.2-compute` (virtio-nvgpu
112eac4) and pushed:
- `ghcr.io/nestrilabs/nestri/base:latest` sha256:152a4485…
- `ghcr.io/nestrilabs/nesbox/jail:latest` sha256:df813051…, rebuilt on
that base from nesbox dev 6d37597, which includes the vhost-user
forwarding
Measured on box2 (RTX 3060, 595.104.02) with virtio-nvgpu's vkrt probe:
with the layer on, a plain device is created and none of the seven
extensions are offered. With it off, all seven are offered and fail with
-3.
On an NVIDIA host the guest's Vulkan driver offers ray tracing, optical
flow and a few others it cannot create a device with, and an engine that
enables them does not start. virtio-nvgpu's layer hides them; it acts
only on a forwarded NVIDIA device, so AMD and Intel boxes are unaffected.
Built from virtio-nvgpu at NVGPU_REF, the ref the kernel build already
takes the guest driver from, and recorded in the image.
The end that reads the first frame of an attach stream to learn whom the
connection is for then forwards the rest untouched, and what arrived behind that
frame in the same read belongs to the stream.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The host can now ask a box for a shell on a pseudo-terminal, the way a container
runtime attaches to a container. `HostToGuest::Attach` starts the process, and
the guest then dials the host on a vsock port of its own (7001) and carries the
terminal's bytes over a small framed stream: data both ways, a resize, a hangup,
and the exit status last. A screenful of output on the JSON-lines control channel
would hold every other message in the box behind it, which is why it is not
there. A failure to start is the exception, and is said on the control channel
because nothing has connected to the data port yet.
A shell is not a launch: it never ends the box, and the box ending ends it. It
is started through the same reaper as everything else, so it cannot become the
zombie that exists to be prevented. With a terminal on the other end the PTY is
ordinary; without one it is put in raw mode so a command run for its output gets
its bytes back unrewritten. Closing the stream hangs the shell up, process group
and all.
The frame codec is a pure bytes-in, frames-out module in nesprotocol with a
payload cap and no skipping of unknown tags, since a length read from a guest is
read across a trust boundary. No version bump: adding a message does not need
one, and a guest that predates it simply never dials.
The runtime stage gains strace, lsof and less, so a root shell in a box is not
one that cannot trace a process.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There is no service manager in a box, so nothing applies the distribution's
sysctl configuration, and the guest ran on the kernel's own defaults. Two of
them are too low for a modern engine under Proton:
- vm.max_map_count defaults to 65530. A large Unreal Engine 5 title maps far
more regions than that, and the failure is a null allocation deep inside the
engine or Wine rather than a message about mappings. It is set to 2147483642,
the value SteamOS and the distributions that package Proton use. It is read
back after the write, because an accepted write that did not take reports
success while the box is still unconfigured.
- RLIMIT_NOFILE is raised on init, which every process in the box inherits. A
child can lower a limit but cannot raise its hard one, so the unprivileged
workload could never get above what it started with. 524288 is the hard limit
Proton's documentation asks for. It is only ever raised.
Both are best effort and each failure says what it costs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The distribution's rust package is built against the distribution's LLVM, and
when the two repositories this image draws from disagree about llvm-libs, rustc
itself fails to start with a symbol-lookup error, before any of our code is
involved and in every stage that builds Rust. rustup's toolchain links nothing
of the distribution's, so `rustup default stable` replaces the package.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`umu-run` is how a title that is not Steam's is launched, so the game fixes
Proton carries apply outside Steam. It is built from a pinned tag in its own
stage rather than installed from the distribution, whose package declares a
long list of lib32 libraries and desktop packages for the Steam Linux Runtime
container this image never runs.
It is built against the distribution's urllib3 and pyzstd instead of the copies
it can vendor: those are built from source with pinned build tooling, and a
rolling distribution moved past a pin (setuptools-scm 10 against a <10
ceiling) and failed the build. The image adds python-xlib, python-urllib3 and
python-pyzstd, none of which pull in anything 32-bit.
Two checks keep it honest. umu-run has to start, because a missing Python module
otherwise shows up as a game that never starts and reads like a script failure;
and no lib32 package may be installed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Polishes the sign-in screens and adds a way to see all of them locally.
**Screens**
- Code fields are drawn as one box per character (device code shows a
dash between halves). One real input sits under the boxes, so typing,
paste and autofill work normally, and with script off it falls back to
the plain field.
- A wrong answer or a refused character shakes the field and rings it
red.
- Focus is white instead of the brand colour; the button stays dim until
the form can be sent.
- One alert at a time: an error replaces "Code sent to …" instead of
joining it.
- The device confirm screen says "A device" instead of the client's
self-chosen name, with the Deny warning moved under the buttons.
- A device code typed with its dash (`BCDF-3467`) is no longer cut to
`BCDF-346`.
**Preview**
`bun run preview` in `apps/auth` serves all 18 screen states on :1338,
at desktop and phone sizes, reloading on save. No database or mail
needed.
To make that possible the issuer's screens moved out of the route
handlers into `packages/auth/src/ui/issuer.ts`, unchanged.
Tests: `bun test` in `packages/auth` (67 pass).
A code field silently dropped a character it does not accept, so typing
a letter into the emailed code, which is digits only, looked exactly
like a field that does not work. It now shakes the boxes and rings them
red, the same answer a wrong code gets. Dashes and spaces are still
taken quietly, since they are how a code is written.
The shake an error page plays on load now rings the field red too,
fading back to its usual border once it settles, on every field
including the email address. Reduced motion keeps the red and drops
the movement, since the red is the message.
The boxes were one input each, with script moving focus between them.
That worked in a clean browser and failed in an ordinary one: on the
emailed code, whose first box is marked as a one-time-code field, typing
and pasting did nothing, while the device code, which is not marked,
worked. A password manager or autofill prompt hooking that field and a
script moving focus between six fields were working against each other.
It also left gaps: pasting into the third box with the second one empty
produced a code with a character missing.
Now the real input is stretched invisibly over the boxes and is the only
thing typed into, pasted into or autofilled; the boxes only display what
it holds, with the next one ringed and showing a caret. With nothing
moving focus there is nothing to fight, and a code cannot have a gap.
The browser's own validation message works again too, since the field
it points at is where the boxes are.
The emailed code and the device code are now typed into a row of boxes,
with a dash drawn between the device code's two halves. Typing moves to
the next box, backspace on an empty box goes back, the arrow keys move,
and a pasted or autofilled code, dashes and all, is spread across them.
It is an enhancement of the field and not a replacement for it. The
server still sends the single plain input, which works on its own with
script off; a short inline script draws the boxes over it and keeps it
in step. The real input is still the one that submits and the one whose
validity dims the button, so nothing else had to learn about boxes.
Pressing the button early moves to the first empty box, since the
browser's own message would point at a field nobody can see.
A device code is shown as BCDF-3467 and will be typed or pasted that
way. The field allowed exactly 8 characters, so a correct code was cut
to BCDF-346 before the server, which ignores separators, ever saw it.
A grouped code's field now has room for its separators and a pattern
that allows a dash or a space between groups, and the device code is
declared as two groups of four.
Every error arrives as a new page, so an animation that runs on load
runs exactly when one does, with no script to trigger it. It plays once,
only on a form showing an error alert, and not at all for anyone who
has asked for reduced motion.
The confirm screen named the client asking to sign in. That name is
chosen by whoever started the request, so it let a stranger's request
introduce itself as anything at all; and to the person reading, it is
just a device. It now says "A device is asking to sign in", and a test
checks that a client's self-chosen name never appears.
The warning about choosing Deny moves under the two buttons, as small
print in the same style as the terms line on the email screen, so the
question and its two answers read first.
bun --watch watches each imported file, and many editors save by
writing a new file and renaming it over the old one. The watch does not
follow the rename, so the first such save reloaded the preview and every
later one was silently ignored.
The preview now runs under a small supervisor that watches the source
directories and restarts the server, which sees a save however it was
made. It also stops the server when it is stopped itself, so the port
is not left held.
A wrong code showed "Invalid code" above "Code sent to ...", two answers
to one question in two colours. An error now replaces the confirmation
instead of joining it.
A focused field was ringed in the brand orange, which reads as an error
and competes with the wordmark. Focus is now grey while the value is
incomplete and white once it would pass, so the field says it is done.
The submit button is dimmed while any field in its form is invalid. It
uses the browser's own validity, so there is still no script, and the
code field gains a pattern so a partly typed code counts as invalid.
The button stays clickable, so pressing it early still gets the
browser's message saying what is missing.
Alerts were a bordered, tinted box with an icon, above the fields. They
are now a line of small text under the fields: red when something went
wrong, green when something just happened, no box and no icon.
The sign-in screens no longer carry the "One place for all the ways you
play" heading. Screens that pass their own heading (device approve,
deny and the rest) keep it.
bun run preview (in apps/auth) serves a gallery of all 18 states the
sign-in flow can show, at desktop and phone sizes, each one also
viewable full size. It needs no database or mail: each state is the
real screen value, drawn by the real renderer with the real theme.
bun --watch restarts the server on any change to the renderer, the
stylesheet or the copy, and every page polls for the restart and
reloads itself, so styling is edit, save, look.
The issuer's screens were written inline in its route handlers, so the
only way to look at one was to reach the condition that produces it: an
expired cookie, a refused grant, ten wrong guesses. They now live in
ui/issuer.ts and the handlers call them, with no change to copy, status
or markup.
The app's theme moves to its own module for the same reason: it can be
used without importing the issuer and its database stores.
The boot descriptor gains `images`: read-only EROFS images mounted on their
own, with no writable layer over them. nesinit mounts them first, before
overlays and shares, with the same nosuid/nodev guards every layer has and
without noexec because what is in one is run. CONTROL_VERSION moves to 5: the
descriptor refuses fields it does not know, and a guest image without Proton is
useless to a host that does not send one, so the mismatch is named at the
handshake.
The guest rootfs no longer contains Proton. The checks that used to guard the
in-image copy (every library its programs and Wine's core link, an interpreter
behind the entry point) move to a compat_check stage that verifies the Proton
named by PROTON_IMAGE and leaves a marker in the image saying which one. The
leaf stages copy that marker out of it so the stage cannot be skipped as
unused. With nothing at a Proton path, a box started without one fails saying
where it looked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
At this point, why not..
Calling it `nespyro` since it's slightly modified, uses slang shader
language instead, along other things..
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The seed was a release behind, so it now matches the tree it is migrated
onto. The scheduler, block-layer and TCP additions the tree ships are pinned
in the fragment, where the build checks them, rather than left to the seed's
defaults. Transparent huge pages move to always, since games map big heaps
without asking for them.
Clang builds the kernel now: whether thin-LTO is offered at all is decided
when Kconfig runs, so LLVM=1 applies to the config steps as well as the
compile, and missing tools fail before anything starts. Allocation and call
path hardening (init-on-alloc, freelist hardening, stack protector) and
debug info are pinned off alongside the speculation mitigations, and the
slab pin the minimal seed needs is restored.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Lets a person put a machine on their team by pasting one command from
the dashboard, instead of copying a user session onto the machine.
## What changes
- **Install tokens** (`packages/core/src/machine/install-token.*`,
migration `0017`). Issued for one team, spent by their first use, expire
after an hour, stored only as a SHA-256 digest. Redeeming is one
conditional `UPDATE` inside the registration transaction: concurrent
redemptions of one token register exactly one machine, and a failed
registration leaves the token unspent.
- **`POST /machine/install-token`** (session): mints a token for a team
the caller belongs to, with the same team resolution and membership rule
as `/register`.
- **`POST /machine/install`** (public): spends a token and returns the
same id, slug and one-time secret as `/register`. Unknown, expired and
used tokens are refused identically.
- **`GET /install.sh`**: a POSIX script embedded at build time. It
checks the platform, asks where box images should live, downloads the
host agent at a pinned version, verifies `SHA256SUMS`, installs to
`~/.local/bin` for the calling user (never root), and hands over with
the token in the environment rather than argv.
- **`GET /install/:component/:version/:asset`**: redirects to a
one-minute signed URL on a private S3-compatible bucket, so every
download passes through a route that can be logged or turned off. The
SigV4 signer uses Web Crypto and is tested against AWS's published
example.
## Configuration
New optional settings: `RELEASES_BUCKET`, `RELEASES_ENDPOINT`,
`RELEASES_REGION`, `RELEASES_ACCESS_KEY_ID`,
`RELEASES_SECRET_ACCESS_KEY`. Without them, downloads return `503` and
everything else is unaffected. Scope the key to reads on that one
bucket.
## Tested
- Core: single use, a five-way race registering exactly one machine,
expired and unknown tokens refused alike.
- Signer: matches the AWS example signature.
- End to end on a real Linux host against a local stack: install script,
signed download from an S3 store that verifies SigV4 (a tampered
signature is refused), registration, and the agent coming online.
GET /install.sh serves a POSIX script, embedded in the API at build time
so the script and the routes that redeem its token ship together. It
checks the platform, asks where box images should live, downloads the
host agent at a pinned version, verifies it against SHA256SUMS, installs
it for the calling user and hands over with the token in the environment
rather than argv.
GET /install/:component/:version/:asset redirects to a one-minute signed
URL on a private S3-compatible bucket, so every download passes through a
route that can be logged or switched off. The SigV4 signer is written
against Web Crypto and checked against AWS's published example.
POST /machine/install-token mints a token for a team the caller belongs
to, with the same team resolution and membership rule as /register.
POST /machine/install needs no session: it spends the token and returns
the same id, slug and one-time secret as registering directly, refusing
unknown, expired and used tokens identically.
Installing on a host should be a command a person pastes, not a user
session copied onto a machine. A token is issued for a team, spent by its
first use, expires after an hour and is stored only as a digest, because
it travels in that command and so lands in shell history.
Redeeming is one conditional update inside the registration transaction:
concurrent redemptions of one token register exactly one machine, and a
registration that fails leaves the token unspent.
Adds `nesgamepad`, basically "diet-coke vimputti" for Nestri's microVM
needs here.
Comes with direct mapping for:
- Sony
- DualShock 4 (v2), DualSense
- Microsoft
- Xbox 360 pad, Xbox One S pad
- Nintendo
- Pro Controller
- Generic pad for unknown controllers
---------
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
"No suitable memory type for DMA-BUF import" names neither mask, so it cannot
tell a driver that refused the descriptor outright (fd_type_bits=0) from one
whose memory types simply do not overlap the image's — which are different
bugs in different components. The three numbers were already computed for a
debug! two lines above, and a debug line only fires at a level nobody runs in
the field; this one did not print at all while a neighbouring crate's did.
Now the failure carries image_type_bits, fd_type_bits, their intersection, and
the size, format, modifier and stride it was importing. On a virtualised NVIDIA
node this turned three unknowns into one measurement and pointed straight at
the kernel side.
The comment explaining the `cmd /C start` bug spelled out the cut submit
link in full. This repository is public and is pull-mirrored onto forges
that crawlers index far more eagerly than GitHub does, so GPTBot found it
there and followed it -- twice. The worker classed both as truncated
submissions, correctly, and raised an alert apiece for a bug fixed in
0.2.1.
Describe the cut URL instead of writing it, and say why, so the next
person documenting a bad link does not paste a working one.
Deliveries arrived intact, verified correctly, and applied to nobody.
Checking the signature here rather than through the SDK means the body is
exactly what was sent, and what is sent is snake_case. The SDK's parser renames
fields to camelCase on the way through, so the reader written against it looked
for `externalId` in a payload that says `external_id` — found nothing, decided
the delivery was about a customer we did not create, and acknowledged it.
That is the worst shape a bug of this kind can take. Every visible signal was
healthy: a 200 back to the provider, no retries, no errors, and a plan that
silently never changed.
Both spellings are now read, so neither path can regress the other.
Registering a team assumed creating a subscription would create the customer it
names. It does not — the customer has to exist first, and creating one fails if
the address is already taken, which it often is: a checkout taken before the
team existed leaves one behind, and so does making one by hand.
So the lookup is now in three steps. A customer already carrying this team id is
used. Otherwise one is found by address and adopted. Otherwise one is made.
**A customer carrying a different team's id is left alone.** Taking it would
move where that subscription is billed, and the team that lost it would go quiet
rather than fail — which is the kind of thing found a month later in a revenue
figure that does not add up.
A team whose owner has no address yet gets no customer, reported rather than
guessed around: an invented address makes a customer nobody can be reached at,
and the next call fixes it once there is a real one.
Checked against the provider rather than only in tests: all four paths — new,
repeated, taken by another team, and no address — behave as written.
Every delivery was refused as a signature mismatch, and nothing in the error
said why.
The provider signs one of two ways. A `whsec_` prefix means Standard Webhooks,
where the secret is a base64 key the verifier decodes; anything else is the
older scheme, where the secret is used as its own raw bytes. Which one a secret
belongs to is decided by when it was created, and every secret created now is
the new one.
The SDK's helper only implements the older scheme — it base64-encodes whatever
it is handed, so a Standard Webhooks secret becomes the literal bytes of the
string including its prefix, and every signature then fails against a key
derived quite differently.
The scheme is now read off the secret rather than configured, so rotating one
cannot put the two out of step. The verifier for the new scheme is the same
library the SDK uses underneath; nothing here hand-rolls crypto.
Tested with a real signature rather than only a rejection. A mismatch is easy to
assert by accident, and a test that only proved bad input is refused would have
passed against the broken version too.
Also teaches the product script to check shape and not just name: a one-time
product where a subscription is wanted cannot be subscribed to at all, and
reporting it as already-there hands back an id that fails at its first use.
Three comments named a file in the internal repo: a contract, a design
guide, and a user-research document with section numbers. A filename plus
a section tells a reader what to ask for, which is the escape the marker
convention exists to prevent.
Each comment now states the requirement on its own terms. The render-node
requirement carries a decision marker; the other two have no decision
behind them, so they cite nothing.
Nine commits, in the order they are easiest to read.
## The shared operator secret is gone
`x-nestri-admin-token` had no caller left — the device pairing it
existed for is
on hold, and nothing in any tree sent it. What remained was a key that
bypassed
authentication entirely and was required to boot.
Every route behind it had a better answer. The two Steam sync routes
took a
`userId` **in the body**, so one secret could write into anybody's
library; they
now authenticate as the host holding that person's Steam sign-in, and
the claim
is checked against the enrolment record. Download-state narrows to hosts
alone.
Creating a game by hand is deleted (syncing already upserts the
catalogue), as is
reading the waitlist — every address on it belongs to someone who has
not agreed
to anything, and answering it over HTTP made that list something a
leaked key
could drain.
Nothing in the API now accepts a credential standing for more than one
caller.
## An organisation owns hardware
Two kinds of machine were modelled as one. A host somebody brings is
theirs,
reached through a team, and should die with their account. A host bought
to serve
other people's workloads is none of those things, and had to be
registered under
an employee's personal team — where their account going away took it
with them.
Ownership is now an either/or, enforced by a check constraint rather
than a
convention: both null is a host nothing can bill, and both set is two
answers to
"whose is this?". Membership of an organisation is derived from a
verified email
domain rather than stored, so signing in with a personal address still
gets an
ordinary personal account.
Not a billing subject. A team pays for what it uses either way.
## Burn, and the three windows
The unit is one second of a reference session, so an allowance is
measured in
time and a bar prints the stored number rather than converting into it.
Counters are stored beside the time they began, and a total whose stamp
has
rolled outside its window reads as zero — so a window clears without
anything
running. No schedule to misfire, no race between a reset and a write.
Two rules on the allowances are enforced rather than remembered: an
allowance
must exceed its own window, or one uninterrupted session hits a wall;
and each
longer one must sit under what the shorter already permits, or it never
binds and
is decoration.
Burn is recorded as segments at one rate, because a run's rate does not
survive
its own lifetime. On our own hardware a bigger tier costs more; on a
caller's own
card it does not, since there is no share of a card of ours being spent.
The gate is at the one moment it may speak — before a run starts, never
again. A
limit refuses the next run and never interrupts one going.
## Payment
Checkout, portal, and a webhook. No price, no currency and no card
detail is
stored: a subscription's existence and its state are the whole of what
crosses
back. Free is a real subscription too, created outright since nothing a
month
needs no payment, so an upgrade changes a subscription rather than
inventing a
customer.
The webhook is the only route no session protects. A signature over the
raw body
stands in for one, checked before the body is parsed, and with no secret
configured it refuses everything.
## Breaking
- `x-nestri-admin-token` is no longer accepted; `ADMIN_SHARED_SECRET` is
no longer read
- `POST /games`, `GET /waitlist` and the `/pairing-code` routes are gone
- `POST /games/sync` and `POST /library/sync` now need host credentials
and take `userId` in the body
- `POST /steam/link` no longer accepts `userId`; `POST
/games/download-state` no longer accepts `hostId`
- `POST /session` responds `{ data, billing }` rather than `{ data }`
## Checks
342 tests pass. Typecheck unchanged from before the branch — the two
pre-existing
errors in `utils/hook.ts` and `utils/validator.ts` are untouched.
Billing is inert until configured: unset `POLAR_*` means every team is
free, no
checkout starts, and the webhook refuses every delivery.
Every team now exists with the payment provider, free ones included. An upgrade
then changes a subscription rather than inventing a customer, and there is one
question to ask about anybody instead of two.
A subscription at nothing a month needs no payment, so it is created outright
rather than by sending somebody through a checkout to pay zero.
It runs after the team rows are committed and cannot affect them. Signing up is
not allowed to depend on a third party being reachable, so this cannot fail the
call and does not retry — a team that misses it is free, which is what it would
have been anyway, and the next call puts it right because the operation is
idempotent.
This broke the webhook mapping, which read the plan off the event type. A free
subscription announces itself with the same `subscription.created` a paid one
does, so every new signup would have landed on the paid allowance. The plan now
comes from the product, and a product we do not sell is left alone rather than
guessed at — somebody selling something else through the same account must not
be able to change what a team may run by doing so.
The external id stays the team. It is the billing subject, and keying on the
user would collapse somebody with two teams into one customer with no way to
say which subscription belonged to which.
Creating a product with one sends `organization_id` alongside a token that
implies it, which is refused outright rather than ignored — so which kind of
token is in hand has to be known before the call, not after. A personal token
can see several organizations and still has to say which.
Checkout, the customer portal, and the webhook that moves a team's plan.
Nothing money-shaped is stored. No price, no currency, no card detail — a
subscription's existence and its state are the whole of what crosses back,
because they are the only two facts the product needs and anything more would
be a second copy of a record somebody else is authoritative for.
Currency is deliberately not ours to hold. A product carries a price per
currency on their side and the customer's location picks one at checkout, so
there is no figure in this API that could drift from the one somebody is
charged. The team id travels as the customer's external id, which keeps the
mapping on their side rather than putting a foreign primary key in our schema.
Access follows their state, and the interesting cases are where that is not the
same as "paying right now". Cancelling keeps the plan: they paid to the end of
the period and turning them off when they click it takes something they bought.
A failed card keeps it too, because a retry that ends in payment should not
have cost them access in the middle. Only a revoked subscription takes it away,
which is the one moment nothing is left that was paid for. An event we do not
recognise changes nothing at all — new types are added by people who do not
know what we do with them, and a default that moved a plan would eventually
cancel an account nobody cancelled.
The webhook is the only route here no session protects, because its caller has
no account and never will. A signature over the raw body stands in for one, and
it is checked before the body is looked at — a body that has been parsed and
re-serialized is not the body that was signed. With no secret configured it
refuses everything rather than accepting anything, since otherwise knowing the
URL would be enough to set somebody's plan. Note also what is absent: no route
sets a plan, so there is no endpoint for granting yourself a subscription.
The product is written down as a definition with a script rather than clicked
into a dashboard, because the two environments are separate servers and nothing
made in one can be moved to the other. Promoting it is running the same script
with the other token, which is the only version of that which cannot drift. It
writes nothing without --apply and refuses to add a second product with a name
already taken.
The rate was a constant. That was correct for everything the system can
currently run and wrong the moment it can run anything else, because a tier
buys a share of a card — so a bigger one on our own hardware is more of
something we bought being spent, and a flat rate there sells a whole card for
the price of a quarter of one.
So a run's rate now comes from what the run is: its size tier, and whose
hardware it sits on.
On the caller's own hardware the tier changes nothing. There is no share of a
card of ours in play, so a run costs one unit a second whatever size it asked
for. Charging somebody more for taking more of a GPU they bought is a tax on
their own hardware, and not doing that is most of what this model is for.
This exposed a bug in what went before. Resegmenting recomputed one shared rate
and wrote it to every open stretch, which was harmless while all runs cost the
same and would have quietly repriced an expensive run as whatever the last one
to start was. Each stretch now keeps its own rate, which is also the more
honest shape: a run's rate is a property of that run, and nothing about it
changed because a sibling appeared or the clock ticked.
The account's total is now the sum of what its runs cost rather than a count
times one rate — an expensive run and a cheap one alongside it are not two of
anything. Concurrency still lands exactly where it did, as there being more to
add, and no run gets dearer because another started.
There is no hardware factor yet and its absence is deliberate: nothing records
which card a host has, so a table keyed on a model would be keyed on nothing.
A faster card should cost more, and that starts with a column.
The reference tier is pinned at exactly one unit a second, checked rather than
assumed. The unit is a second of a reference session, so moving it would
silently redefine every allowance — the same stored number would mean a
different number of hours.
The counters this fills are the ones the windows already knew how to read.
What was missing was anything that put a number in them.
Burn is recorded as segments: one stretch of one run at one unchanging rate,
opened when the rate becomes true and closed when it stops being. Not a row per
session, because a session's rate does not survive its own lifetime — a second
run changes what the account spends per second while the first is still going,
and a rate that applied from that moment must not be backdated over the time
before it. Not a row per event either, because burn accrues against an envelope
that is held rather than per thing consumed.
Closing a segment is what moves burn into the counters, so a long run lands
incrementally instead of all at the end. Burn that only arrives when a session
stops is burn that cannot refuse the next one, and a bar that does not move
while something is running is a bar nobody believes.
The counters are written with the staleness rule as a single statement: add to
the total if its stamp is still inside the window, otherwise start again from
this amount. Reading and then deciding would be two statements with a gap, and
the gap is where a concurrent tick doubles or vanishes. The first tick for a
team and the thousandth are the same call, for the same reason.
The gate sits at the one moment it is allowed to speak — before a run starts,
never again. A limit refuses the next run and never interrupts one already
going; someone losing a session mid-game to a meter does not come back. Every
window is checked rather than the shortest, because they protect different
things over different spans.
The answer comes back with the created run rather than being thrown away: the
response carries where each window stands, what the account spends per second
now, and what one more run would cost. Every surface that can start a run has
to show that before the click, and a second call for it is a call nobody makes.
Asking twice would also let the number shown and the number billed disagree.
Accrual is wired to the run's own state transition, in the same transaction
that moves it. A session that went live without its meter starting is free
hardware; one that ended without its meter stopping bills forever. Both are
silent, so neither may be a second write that might not happen.
The unit is one second of a reference session — baseline size, baseline card,
running alone, on hardware we own. Every factor is a multiple of that, so an
allowance is measured in time and a bar prints the stored number instead of
converting into it. Integers throughout.
Three rolling windows, one function. A counter is stored beside the time it was
last written, and a counter whose timestamp falls outside its window reads as
zero — so the reset is implied by the clock and nothing has to run for a window
to roll clear. No scheduled job to misfire, and no race between a reset and a
write landing together. The same rule on the write side is one statement rather
than a read followed by a decision.
The check is pure, and it is the same arithmetic the meter draws from. The
complaint about usage limits is almost never the limit, it is being surprised
by one, and two implementations that agree today are how a full bar and a
refusal start disagreeing.
Two rules on the allowances, enforced rather than remembered:
- A window's allowance must exceed the window itself. Because the windows roll,
one uninterrupted session asymptotes at exactly the window length, so an
allowance at or below it is a wall that someone playing alone will meet.
- Each longer allowance must be under what the shorter window already permits,
or it can never be reached — a number that looks like a limit, reads like a
promise, and never once fires.
Allowances are configuration rather than constants, because they will be
retuned against real burn far more often than this code changes, and a rate
that needs a deploy is a rate that stays wrong until the next one. The shipped
set is explicitly a placeholder: coherent enough to test against, not a pricing
decision.
DatCaptainHorse
Wanjohi
