mirror of
https://github.com/nestriness/nestri.git
synced 2026-11-03 22:22:39 +02:00
prod-38b8a43
478
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Wanjohi
|
38b8a43a1b |
Release: host install, billing fixes, nvgpu guest, media bitrate (#352)
Promotes `dev` to `prod`: install tokens with `/install.sh` and signed downloads (#351), controller support (#350), the nvgpu guest (#349), media bitrate control and HDR (#346), and three billing fixes. Includes migration `0017_install_token`.prod-38b8a43 |
||
|
Wanjohi
|
b281cb102e |
feat: Install a host with one pasted command (#351)
Lets a person put a machine on their team by pasting one command from the dashboard, instead of copying a user session onto the machine. ## What changes - **Install tokens** (`packages/core/src/machine/install-token.*`, migration `0017`). Issued for one team, spent by their first use, expire after an hour, stored only as a SHA-256 digest. Redeeming is one conditional `UPDATE` inside the registration transaction: concurrent redemptions of one token register exactly one machine, and a failed registration leaves the token unspent. - **`POST /machine/install-token`** (session): mints a token for a team the caller belongs to, with the same team resolution and membership rule as `/register`. - **`POST /machine/install`** (public): spends a token and returns the same id, slug and one-time secret as `/register`. Unknown, expired and used tokens are refused identically. - **`GET /install.sh`**: a POSIX script embedded at build time. It checks the platform, asks where box images should live, downloads the host agent at a pinned version, verifies `SHA256SUMS`, installs to `~/.local/bin` for the calling user (never root), and hands over with the token in the environment rather than argv. - **`GET /install/:component/:version/:asset`**: redirects to a one-minute signed URL on a private S3-compatible bucket, so every download passes through a route that can be logged or turned off. The SigV4 signer uses Web Crypto and is tested against AWS's published example. ## Configuration New optional settings: `RELEASES_BUCKET`, `RELEASES_ENDPOINT`, `RELEASES_REGION`, `RELEASES_ACCESS_KEY_ID`, `RELEASES_SECRET_ACCESS_KEY`. Without them, downloads return `503` and everything else is unaffected. Scope the key to reads on that one bucket. ## Tested - Core: single use, a five-way race registering exactly one machine, expired and unknown tokens refused alike. - Signer: matches the AWS example signature. - End to end on a real Linux host against a local stack: install script, signed download from an S3 store that verifies SigV4 (a tampered signature is refused), registration, and the agent coming online. |
||
|
Wanjohi
|
065af689b3 |
feat(api): serve the host installer and its downloads
GET /install.sh serves a POSIX script, embedded in the API at build time so the script and the routes that redeem its token ship together. It checks the platform, asks where box images should live, downloads the host agent at a pinned version, verifies it against SHA256SUMS, installs it for the calling user and hands over with the token in the environment rather than argv. GET /install/:component/:version/:asset redirects to a one-minute signed URL on a private S3-compatible bucket, so every download passes through a route that can be logged or switched off. The SigV4 signer is written against Web Crypto and checked against AWS's published example. |
||
|
Wanjohi
|
54dbe8628a |
feat(api): issue and redeem install tokens
POST /machine/install-token mints a token for a team the caller belongs to, with the same team resolution and membership rule as /register. POST /machine/install needs no session: it spends the token and returns the same id, slug and one-time secret as registering directly, refusing unknown, expired and used tokens identically. |
||
|
Wanjohi
|
f691b56c0c |
feat(core): one-time install tokens that register a host to a team
Installing on a host should be a command a person pastes, not a user session copied onto a machine. A token is issued for a team, spent by its first use, expires after an hour and is stored only as a digest, because it travels in that command and so lands in shell history. Redeeming is one conditional update inside the registration transaction: concurrent redemptions of one token register exactly one machine, and a registration that fails leaves the token unspent. |
||
  
|
451b495de6 |
feat: controller/gamepad support (#350)
Adds `nesgamepad`, basically "diet-coke vimputti" for Nestri's microVM needs here. Comes with direct mapping for: - Sony - DualShock 4 (v2), DualSense - Microsoft - Xbox 360 pad, Xbox One S pad - Nintendo - Pro Controller - Generic pad for unknown controllers --------- Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
291fabd145 |
feat: nvgpu guest (#349)
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0811f57f1a |
feat: media bitrate control, HDR (#346)
Fixes: #335 Still a work-in-progress. --------- Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Wanjohi <elviswanjohi47@gmail.com> |
||
|
Wanjohi
|
1c721962f4 |
fix(nescapture): put the masks in the DMA-BUF import failure
"No suitable memory type for DMA-BUF import" names neither mask, so it cannot tell a driver that refused the descriptor outright (fd_type_bits=0) from one whose memory types simply do not overlap the image's — which are different bugs in different components. The three numbers were already computed for a debug! two lines above, and a debug line only fires at a level nobody runs in the field; this one did not print at all while a neighbouring crate's did. Now the failure carries image_type_bits, fd_type_bits, their intersection, and the size, format, modifier and stride it was importing. On a virtualised NVIDIA node this turned three unknowns into one measurement and pointed straight at the kernel side. |
||
|
Wanjohi
|
3482ed6788 |
docs(nesdoctor): a live URL in a comment is a URL something will follow
The comment explaining the `cmd /C start` bug spelled out the cut submit link in full. This repository is public and is pull-mirrored onto forges that crawlers index far more eagerly than GitHub does, so GPTBot found it there and followed it -- twice. The worker classed both as truncated submissions, correctly, and raised an alert apiece for a bug fixed in 0.2.1. Describe the cut URL instead of writing it, and say why, so the next person documenting a bad link does not paste a working one. |
||
|
Wanjohi
|
8284d9f1d3 |
Fix webhook payload field names (#345)
Two fixes, both found by testing against the provider rather than in isolation. **Deliveries applied to nobody.** Verifying the signature directly means the body is exactly what was sent — snake_case — while the reader was written against the SDK parser's camelCase. So every delivery arrived, verified, matched no customer, and was acknowledged. Every visible signal was healthy: 200s, no retries, no errors, and a plan that never changed. Both spellings are now read. **A customer may already exist.** Registering a team assumed creating a subscription would create the customer it names; it does not, and creating one fails when the address is taken. Now: use the customer already carrying this team id, else adopt one found by address, else make one. A customer carrying a *different* team's id is left alone — taking it would move where that subscription is billed, and the team that lost it would go quiet rather than fail. All four paths checked against the live sandbox: new, repeated, taken by another team, and no address. 346 tests pass.prod-8284d9f |
||
|
Wanjohi
|
a09ad1e09a |
fix(billing): read the payload that was actually sent
Deliveries arrived intact, verified correctly, and applied to nobody. Checking the signature here rather than through the SDK means the body is exactly what was sent, and what is sent is snake_case. The SDK's parser renames fields to camelCase on the way through, so the reader written against it looked for `externalId` in a payload that says `external_id` — found nothing, decided the delivery was about a customer we did not create, and acknowledged it. That is the worst shape a bug of this kind can take. Every visible signal was healthy: a 200 back to the provider, no retries, no errors, and a plan that silently never changed. Both spellings are now read, so neither path can regress the other. |
||
|
Wanjohi
|
abd27f2d5f |
fix(billing): a customer may already exist, and may belong to somebody else
Registering a team assumed creating a subscription would create the customer it names. It does not — the customer has to exist first, and creating one fails if the address is already taken, which it often is: a checkout taken before the team existed leaves one behind, and so does making one by hand. So the lookup is now in three steps. A customer already carrying this team id is used. Otherwise one is found by address and adopted. Otherwise one is made. **A customer carrying a different team's id is left alone.** Taking it would move where that subscription is billed, and the team that lost it would go quiet rather than fail — which is the kind of thing found a month later in a revenue figure that does not add up. A team whose owner has no address yet gets no customer, reported rather than guessed around: an invented address makes a customer nobody can be reached at, and the next call fixes it once there is a real one. Checked against the provider rather than only in tests: all four paths — new, repeated, taken by another team, and no address — behave as written. |
||
|
Wanjohi
|
8c3e460312 |
Fix webhook signature verification (#344)
Two commits. **The webhook fix.** Every delivery was being refused as a signature mismatch. The provider signs one of two ways and the SDK helper only implements the older one, so a `whsec_`-prefixed secret — which is what any secret created now is — was mangled before verification. The scheme is now read off the secret itself, so rotating one cannot put the two out of step. Verified against a real signature, not only against a rejection. **A leak fix that was never pushed.** `apps/nesdoctor/src/sys.rs` and `ask.rs` still cite an internal document by filename in comments that reach a public repo. The commit fixing it existed only on a local branch. Rebased on and included here; the third file it touched had already been fixed differently. 344 tests pass.prod-8c3e460 |
||
|
Wanjohi
|
5b90bd0f0c |
fix(billing): verify the signing scheme the secret actually says it is
Every delivery was refused as a signature mismatch, and nothing in the error said why. The provider signs one of two ways. A `whsec_` prefix means Standard Webhooks, where the secret is a base64 key the verifier decodes; anything else is the older scheme, where the secret is used as its own raw bytes. Which one a secret belongs to is decided by when it was created, and every secret created now is the new one. The SDK's helper only implements the older scheme — it base64-encodes whatever it is handed, so a Standard Webhooks secret becomes the literal bytes of the string including its prefix, and every signature then fails against a key derived quite differently. The scheme is now read off the secret rather than configured, so rotating one cannot put the two out of step. The verifier for the new scheme is the same library the SDK uses underneath; nothing here hand-rolls crypto. Tested with a real signature rather than only a rejection. A mismatch is easy to assert by accident, and a test that only proved bad input is refused would have passed against the broken version too. Also teaches the product script to check shape and not just name: a one-time product where a subscription is wanted cannot be subscribed to at all, and reporting it as already-there hands back an id that fails at its first use. |
||
|
Wanjohi
|
2dfb7f4007 |
docs: stop citing internal documents from public comments
Three comments named a file in the internal repo: a contract, a design guide, and a user-research document with section numbers. A filename plus a section tells a reader what to ask for, which is the escape the marker convention exists to prevent. Each comment now states the requirement on its own terms. The render-node requirement carries a decision marker; the other two have no decision behind them, so they cite nothing. |
||
|
Wanjohi
|
de325792a2 |
Release: billing, burn limits and organisations (#343)
Promotes #342 to production. Merging this cuts a release onto the machine. ## What changes for anything already running **`x-nestri-admin-token` stops working**, and `ADMIN_SHARED_SECRET` is no longer read. Nothing in any tree sends it, and the machine's `api.env` still carries one — it becomes an unused line, harmless but worth deleting. **Routes that disappear:** `POST /games`, `GET /waitlist`, and all of `/pairing-code`. The pairing-code core module and its table stay, so returning to that flow is a route file rather than a migration. **Routes that change shape:** `POST /games/sync` and `POST /library/sync` now require host credentials and take `userId` in the body; `POST /steam/link` no longer accepts `userId`; `POST /games/download-state` no longer accepts `hostId`; `POST /session` responds `{ data, billing }` rather than `{ data }`. ## Migrations Two, applied by the release before the swap: - `0015` — the organisation table, and machine ownership becomes team-or-org with a check constraint. Safe in one step: every existing row has a team and no organisation, so all of them already satisfy it. - `0016` — burn counters and rate segments. Two new tables, nothing altered. ## Billing is inert until it is configured With `POLAR_*` unset — which is how the machine will come up — every team is free, no checkout can start, and the webhook refuses every delivery. That is a deployment without billing rather than a broken one, so this can land before the provider settings exist. Filling them in needs, in order: the product created in the production Polar account, a webhook endpoint pointing at the API's public `/billing/webhook`, and the four values written to `/etc/nestri/api.env`. ## Rollback The previous release stays on the machine and the symlink swaps back. The migrations do not roll back with it: `0015` and `0016` only add, and `0015` relaxes two NOT NULLs, so the prior binary runs against the newer schema without noticing.prod-de32579 |
||
|
Wanjohi
|
916473bbbd |
Billing: burn windows, organisations, and payment (#342)
Nine commits, in the order they are easiest to read.
## The shared operator secret is gone
`x-nestri-admin-token` had no caller left — the device pairing it
existed for is
on hold, and nothing in any tree sent it. What remained was a key that
bypassed
authentication entirely and was required to boot.
Every route behind it had a better answer. The two Steam sync routes
took a
`userId` **in the body**, so one secret could write into anybody's
library; they
now authenticate as the host holding that person's Steam sign-in, and
the claim
is checked against the enrolment record. Download-state narrows to hosts
alone.
Creating a game by hand is deleted (syncing already upserts the
catalogue), as is
reading the waitlist — every address on it belongs to someone who has
not agreed
to anything, and answering it over HTTP made that list something a
leaked key
could drain.
Nothing in the API now accepts a credential standing for more than one
caller.
## An organisation owns hardware
Two kinds of machine were modelled as one. A host somebody brings is
theirs,
reached through a team, and should die with their account. A host bought
to serve
other people's workloads is none of those things, and had to be
registered under
an employee's personal team — where their account going away took it
with them.
Ownership is now an either/or, enforced by a check constraint rather
than a
convention: both null is a host nothing can bill, and both set is two
answers to
"whose is this?". Membership of an organisation is derived from a
verified email
domain rather than stored, so signing in with a personal address still
gets an
ordinary personal account.
Not a billing subject. A team pays for what it uses either way.
## Burn, and the three windows
The unit is one second of a reference session, so an allowance is
measured in
time and a bar prints the stored number rather than converting into it.
Counters are stored beside the time they began, and a total whose stamp
has
rolled outside its window reads as zero — so a window clears without
anything
running. No schedule to misfire, no race between a reset and a write.
Two rules on the allowances are enforced rather than remembered: an
allowance
must exceed its own window, or one uninterrupted session hits a wall;
and each
longer one must sit under what the shorter already permits, or it never
binds and
is decoration.
Burn is recorded as segments at one rate, because a run's rate does not
survive
its own lifetime. On our own hardware a bigger tier costs more; on a
caller's own
card it does not, since there is no share of a card of ours being spent.
The gate is at the one moment it may speak — before a run starts, never
again. A
limit refuses the next run and never interrupts one going.
## Payment
Checkout, portal, and a webhook. No price, no currency and no card
detail is
stored: a subscription's existence and its state are the whole of what
crosses
back. Free is a real subscription too, created outright since nothing a
month
needs no payment, so an upgrade changes a subscription rather than
inventing a
customer.
The webhook is the only route no session protects. A signature over the
raw body
stands in for one, checked before the body is parsed, and with no secret
configured it refuses everything.
## Breaking
- `x-nestri-admin-token` is no longer accepted; `ADMIN_SHARED_SECRET` is
no longer read
- `POST /games`, `GET /waitlist` and the `/pairing-code` routes are gone
- `POST /games/sync` and `POST /library/sync` now need host credentials
and take `userId` in the body
- `POST /steam/link` no longer accepts `userId`; `POST
/games/download-state` no longer accepts `hostId`
- `POST /session` responds `{ data, billing }` rather than `{ data }`
## Checks
342 tests pass. Typecheck unchanged from before the branch — the two
pre-existing
errors in `utils/hook.ts` and `utils/validator.ts` are untouched.
Billing is inert until configured: unset `POLAR_*` means every team is
free, no
checkout starts, and the webhook refuses every delivery.
|
||
|
Wanjohi
|
60c4f61bfa |
feat(billing): free is a subscription too, and the product says which plan
Every team now exists with the payment provider, free ones included. An upgrade then changes a subscription rather than inventing a customer, and there is one question to ask about anybody instead of two. A subscription at nothing a month needs no payment, so it is created outright rather than by sending somebody through a checkout to pay zero. It runs after the team rows are committed and cannot affect them. Signing up is not allowed to depend on a third party being reachable, so this cannot fail the call and does not retry — a team that misses it is free, which is what it would have been anyway, and the next call puts it right because the operation is idempotent. This broke the webhook mapping, which read the plan off the event type. A free subscription announces itself with the same `subscription.created` a paid one does, so every new signup would have landed on the paid allowance. The plan now comes from the product, and a product we do not sell is left alone rather than guessed at — somebody selling something else through the same account must not be able to change what a team may run by doing so. The external id stays the team. It is the billing subject, and keying on the user would collapse somebody with two teams into one customer with no way to say which subscription belonged to which. |
||
|
Wanjohi
|
b01d8eabb3 |
fix(billing): an organization token already names its organization
Creating a product with one sends `organization_id` alongside a token that implies it, which is refused outright rather than ignored — so which kind of token is in hand has to be known before the call, not after. A personal token can see several organizations and still has to say which. |
||
|
Wanjohi
|
757ff79233 |
feat(core,api): take payment, and let the provider decide who is paid up
Checkout, the customer portal, and the webhook that moves a team's plan. Nothing money-shaped is stored. No price, no currency, no card detail — a subscription's existence and its state are the whole of what crosses back, because they are the only two facts the product needs and anything more would be a second copy of a record somebody else is authoritative for. Currency is deliberately not ours to hold. A product carries a price per currency on their side and the customer's location picks one at checkout, so there is no figure in this API that could drift from the one somebody is charged. The team id travels as the customer's external id, which keeps the mapping on their side rather than putting a foreign primary key in our schema. Access follows their state, and the interesting cases are where that is not the same as "paying right now". Cancelling keeps the plan: they paid to the end of the period and turning them off when they click it takes something they bought. A failed card keeps it too, because a retry that ends in payment should not have cost them access in the middle. Only a revoked subscription takes it away, which is the one moment nothing is left that was paid for. An event we do not recognise changes nothing at all — new types are added by people who do not know what we do with them, and a default that moved a plan would eventually cancel an account nobody cancelled. The webhook is the only route here no session protects, because its caller has no account and never will. A signature over the raw body stands in for one, and it is checked before the body is looked at — a body that has been parsed and re-serialized is not the body that was signed. With no secret configured it refuses everything rather than accepting anything, since otherwise knowing the URL would be enough to set somebody's plan. Note also what is absent: no route sets a plan, so there is no endpoint for granting yourself a subscription. The product is written down as a definition with a script rather than clicked into a dashboard, because the two environments are separate servers and nothing made in one can be moved to the other. Promoting it is running the same script with the other token, which is the only version of that which cannot drift. It writes nothing without --apply and refuses to add a second product with a name already taken. |
||
|
Wanjohi
|
4c22586d59 |
feat(core): price a run by the size it holds, on hardware we pay for
The rate was a constant. That was correct for everything the system can currently run and wrong the moment it can run anything else, because a tier buys a share of a card — so a bigger one on our own hardware is more of something we bought being spent, and a flat rate there sells a whole card for the price of a quarter of one. So a run's rate now comes from what the run is: its size tier, and whose hardware it sits on. On the caller's own hardware the tier changes nothing. There is no share of a card of ours in play, so a run costs one unit a second whatever size it asked for. Charging somebody more for taking more of a GPU they bought is a tax on their own hardware, and not doing that is most of what this model is for. This exposed a bug in what went before. Resegmenting recomputed one shared rate and wrote it to every open stretch, which was harmless while all runs cost the same and would have quietly repriced an expensive run as whatever the last one to start was. Each stretch now keeps its own rate, which is also the more honest shape: a run's rate is a property of that run, and nothing about it changed because a sibling appeared or the clock ticked. The account's total is now the sum of what its runs cost rather than a count times one rate — an expensive run and a cheap one alongside it are not two of anything. Concurrency still lands exactly where it did, as there being more to add, and no run gets dearer because another started. There is no hardware factor yet and its absence is deliberate: nothing records which card a host has, so a table keyed on a model would be keyed on nothing. A faster card should cost more, and that starts with a column. The reference tier is pinned at exactly one unit a second, checked rather than assumed. The unit is a second of a reference session, so moving it would silently redefine every allowance — the same stored number would mean a different number of hours. |
||
|
Wanjohi
|
b4776fad2f |
feat(core,api): record burn as rate segments, and refuse the next run when spent
The counters this fills are the ones the windows already knew how to read. What was missing was anything that put a number in them. Burn is recorded as segments: one stretch of one run at one unchanging rate, opened when the rate becomes true and closed when it stops being. Not a row per session, because a session's rate does not survive its own lifetime — a second run changes what the account spends per second while the first is still going, and a rate that applied from that moment must not be backdated over the time before it. Not a row per event either, because burn accrues against an envelope that is held rather than per thing consumed. Closing a segment is what moves burn into the counters, so a long run lands incrementally instead of all at the end. Burn that only arrives when a session stops is burn that cannot refuse the next one, and a bar that does not move while something is running is a bar nobody believes. The counters are written with the staleness rule as a single statement: add to the total if its stamp is still inside the window, otherwise start again from this amount. Reading and then deciding would be two statements with a gap, and the gap is where a concurrent tick doubles or vanishes. The first tick for a team and the thousandth are the same call, for the same reason. The gate sits at the one moment it is allowed to speak — before a run starts, never again. A limit refuses the next run and never interrupts one already going; someone losing a session mid-game to a meter does not come back. Every window is checked rather than the shortest, because they protect different things over different spans. The answer comes back with the created run rather than being thrown away: the response carries where each window stands, what the account spends per second now, and what one more run would cost. Every surface that can start a run has to show that before the click, and a second call for it is a call nobody makes. Asking twice would also let the number shown and the number billed disagree. Accrual is wired to the run's own state transition, in the same transaction that moves it. A session that went live without its meter starting is free hardware; one that ended without its meter stopping bills forever. Both are silent, so neither may be a second write that might not happen. |
||
|
Wanjohi
|
b819367a09 |
feat(core): burn windows, and the rules that make an allowance mean something
The unit is one second of a reference session — baseline size, baseline card, running alone, on hardware we own. Every factor is a multiple of that, so an allowance is measured in time and a bar prints the stored number instead of converting into it. Integers throughout. Three rolling windows, one function. A counter is stored beside the time it was last written, and a counter whose timestamp falls outside its window reads as zero — so the reset is implied by the clock and nothing has to run for a window to roll clear. No scheduled job to misfire, and no race between a reset and a write landing together. The same rule on the write side is one statement rather than a read followed by a decision. The check is pure, and it is the same arithmetic the meter draws from. The complaint about usage limits is almost never the limit, it is being surprised by one, and two implementations that agree today are how a full bar and a refusal start disagreeing. Two rules on the allowances, enforced rather than remembered: - A window's allowance must exceed the window itself. Because the windows roll, one uninterrupted session asymptotes at exactly the window length, so an allowance at or below it is a wall that someone playing alone will meet. - Each longer allowance must be under what the shorter window already permits, or it can never be reached — a number that looks like a limit, reads like a promise, and never once fires. Allowances are configuration rather than constants, because they will be retuned against real burn far more often than this code changes, and a rate that needs a deploy is a rate that stays wrong until the next one. The shipped set is explicitly a placeholder: coherent enough to test against, not a pricing decision. |
||
|
Wanjohi
|
15631f5d25 |
feat(core,api): an organisation owns hardware, and a domain says who belongs
Two kinds of machine were modelled as one. A host somebody brings is theirs, reached through a team, and should die with their account. A host bought to serve other people's workloads is none of those things — and there was nowhere to put it, so it had to be registered under an employee's personal team, where it was that person's property and their account going away took it with them. Ownership becomes an either/or. A machine names a team or an organisation, exactly one, enforced by a check constraint rather than by convention: both null is a host nothing can bill, and both set is two answers to "whose is this?" where whichever join a query happens to take decides who pays. Hardware an organisation owns has no team and no person at all, which is the point. The organisation is deliberately not a billing subject and has no plan columns. It says who owns the metal; a team pays for what it uses either way. Membership is derived from a verified email domain rather than stored. An address is already the root identity, so a second record of who belongs where is a second answer that can disagree with the first — and deriving it means signing in with a personal address still gets an ordinary personal account, which is what lets one person hold a company account and use the consumer product. Nothing is granted on an unverified domain or an unverified address: either one is a string somebody typed. Entitlement on fleet hardware refuses everyone for now, with a reason that says so. What grants a run on metered hardware is a plan, and there is nothing to ask yet, so it fails closed rather than giving the expensive case away. The branch is written out so the plan check has one obvious place to land. Routes are read-only, and nothing seeds an organisation. Creating one grants membership to everyone who can receive mail at a domain, so it is an operator action against the database — a migration that inserted one would insert it into every deployment, including ones we have nothing to do with. See docs/deploy.md. |
||
|
|
c2ede3cdba |
feat: nescapture instrumentation, custom Mesa patches (#341)
Whoops, forgot to push these ones.. --------- Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Wanjohi
|
4a2a4412c2 |
style: apply the formatter across the tree
The formatter had never been run over these files, so `oxfmt` on a couple of touched files rewrote two dozen others and buried the real change. Running it everywhere once makes the next diff mean something. No behaviour changes: import ordering, line joining, and reindented JSON in the generated migration snapshots. Both test suites and both typechecks give the same answers as before, including the two type errors this does not fix. |
||
|
Wanjohi
|
40b4270161 |
refactor(api)!: remove the shared operator secret, and let hosts sync their own
A single secret that turned any request into an operator was the only credential several routes accepted, and it had no caller left: the device pairing it existed for is on hold, and nothing in this tree or any client sent it. What remained was a key that bypassed authentication entirely, required to boot, and checked by nobody. Every route behind it had a better answer available: - Library and game sync move to host credentials. Both took a `userId` in the body, which meant one secret could write into anybody's library. A host now says which of its enrolled users a batch is for, and that claim is checked against the Steam sign-ins it actually holds — one box carries several people's accounts, so the pair is the unit. - Download-state reporting narrows to hosts alone, and the body that could name a different host is gone. Which host is reporting comes from its own credentials, and a body that still names one is refused rather than ignored. - Linking a Steam account is always for the caller. - Creating a game by hand is deleted; syncing already upserts the catalogue. - Reading the waitlist is deleted. Every address on it belongs to someone who has not agreed to anything, and answering it over HTTP made that list something a leaked key could drain. - The pairing-code routes are deleted with the flow they served. The domain module and its table stay, so returning to it is a route file rather than a migration. Nothing in the API now accepts a credential that stands for more than one caller: every request resolves to a specific user or a specific host, which is what lets a route say "the caller's own library" and mean it. BREAKING CHANGE: the `x-nestri-admin-token` header is no longer accepted and `ADMIN_SHARED_SECRET` is no longer read. `POST /games`, `GET /waitlist` and the `/pairing-code` routes are gone; `POST /games/sync` and `POST /library/sync` now require host credentials and take `userId` in the body; `POST /steam/link` no longer accepts `userId`; `POST /games/download-state` no longer accepts `hostId`. |
||
|
|
6811c93d51 |
feat: nescapture capture improvements and drive mounts (#337)
Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Wanjohi
|
2924e113ca |
feat(machine): mint a public name, and stop routing on the id (#340)
## Why
The first machine routed end to end earlier today on
`mch_0b10cc….nestri.link`. It worked, and the hostname should not be the
primary key.
Three problems, only the first cosmetic:
- **Ids are monotonic.** `Identifier.ascending` sorts by creation time,
so a public hostname discloses roughly when a machine was registered and
where it falls among its owner's others.
- **An id cannot be rotated.** A name that has been scraped or shared
with the wrong person could only be changed by re-registering the
machine — changing its identity to fix its name.
- **The hostname is also the OAuth audience.** The 302 this edge serves
carries `client_id=mch_0b10cc…`, so the id reaches redirect URLs,
browser history and every access log on the way to the issuer.
Settled in `edge-routing.md` §1.3 before any code was written.
## What changed
`machine.slug` — two words and four digits (`amber-otter-4821`), unique
across the fleet, minted at registration and returned to the host.
DNS-safe by construction, which an id was not: an id carries `_`, which
is not a legal hostname character and works only until it meets a
stricter resolver or TLS stack than the one it was tested against.
Words rather than random characters because the people reading one aloud
are operators on a call. The list is curated, not a dictionary dump —
every pair is shown to strangers.
Reserved labels (`edge`, `api`, `auth`, …) are refused at mint time, not
filtered per request. A machine minted onto `edge` would take the
published-key path away from every host in the fleet at once, and a
request-time filter leaves the bad row in the database.
Minting retries on the unique index rather than checking first: two
registrations in the same instant both read "free" and both write, so
the index is the thing that actually decides. Only a name collision
retries — a duplicate id or secret means something a new name cannot
fix.
## The migration is hand-written, and here is why
`drizzle-kit generate` produced `ALTER TABLE "machine" ADD COLUMN "slug"
text NOT NULL;`. Against a populated table:
```
ERROR: column "slug" of relation "machine" contains null values
```
A default would be worse than the failure — every existing row would
take the same value, the unique index would reject them, and a shared
value on a routing key must not exist even for an instant. So: nullable,
backfill, then require.
Verified against a database at `0013` with three machine rows, applying
`0014`:
```
ALTER TABLE
UPDATE 3
ALTER TABLE
CREATE INDEX
id | slug
--------------------------------+------------------
mch_0b10cc1bd001yby6nq5atKThz5 | host-a0fd57-7631
mch_0b10cc1bd001yby6nq5atKThz6 | host-e60922-2988
mch_0b10cc1bd001yby6nq5atKThz7 | host-e03632-9192
```
All 14 migrations also apply cleanly to an empty database.
The backfill is deterministic (`md5(id)`), so re-running produces the
same names rather than a second set. Backfilled names are `host-…` and
are not minted names — they exist so the constraint can be applied.
## Paired change
`nescore` on `dev` moves the edge's routing query from `m."id"` to
`m."slug"`. **Both must be deployed for routing to keep working** — the
edge selects a column this migration creates. Order: this PR (migration
runs before the swap), then the edge.
Boxes still key on their id, because a box has no name and nothing dials
one. Safe only while one of the two spaces is unreachable — flagged in
the code and in §1.3: both must share a single unique index before a box
serves anything.
## What this does not verify
- **No browser has been through the renamed flow.** The audience changes
with the hostname, so the sign-in redirect now carries the slug as
`client_id`. That path has not been walked end to end since the change.
- **The wordlist has had one pass, not a review.** It is chosen so no
pair reads as a slur or a person, but that is my judgement and it is the
kind of thing worth a second reader before strangers see generated
names.
- **`nestripc-1` will land on a backfilled `host-…` name**, not a minted
one. It should be re-minted after deploy; there is exactly one row and
no route depends on the old id once the edge ships.
- The API's type errors (10) are unchanged and pre-existing;
`packages/core` likewise.
prod-2924e11
|
||
|
Wanjohi
|
ebc0242b49 | Merge branch 'prod' into dev | ||
|
Wanjohi
|
04210cc9f1 |
refactor(auth): describe sign-in screens as data, draw them in one place (#339)
Providers stop returning `Response`. Each says what it needs from the
person — an address, a pin, a yes-or-no — as a `Screen`, and one
`Renderer` draws it.
The old shape made every provider a small web framework: it had to know
about markup, about the stylesheet's attribute names, about how a page
is assembled. So each grew its own callback signature, its own copy, and
its own `new Response(jsx.toString())` — and there was no shared
vocabulary left to style.
### What that had already cost
- **The device flow never got a design.** Its two pages were built by
concatenating HTML strings, with an inline `style` on the user code. Six
more replies were `text/plain` — including the one a person gets when
their sign-in cookie expires.
- **The password screens were already broken.** They render
`data-component="input"`, `data-component="link"`,
`data-component="form-footer"` against a stylesheet that was rewritten
for the code flow. Nobody noticed because password sign-in is not
switched on. They are deleted here, not repaired.
- **A provider could not be named or marked without editing the
library.** `DiscordProvider` has existed all along; wiring it up
rendered **"Continue with discord"**, lowercase, no icon, because the
marks and display names were two hardcoded `const` records inside the
code that drew the chooser.
### What changed
`ui/screen.ts` — four screen kinds (`choose`, `form`, `confirm`,
`message`) and a field vocabulary, as plain data. No JSX, no hono.
`ui/render.tsx` — the only file that knows what a button looks like.
`Renderer` is one method, so replacing the presentation layer wholesale
means implementing that and nothing else.
`Provider.display` — each provider carries its own name and mark
(`ui/mark.ts`, raw SVG strings so `provider/*.ts` never imports a
rendering library). The chooser is built from what providers declare;
`issuer({ chooser })` is left with only the two decisions a deployment
makes that a provider cannot — whether to offer it, and what to put
first.
The theme global is gone. It was `globalThis`, with a comment conceding
as much: every component depended on something invisible at the call
site, untestable in isolation, and shared mutable state on a runtime
that keeps one module instance across requests. It is a closure argument
now, and `Theme` shrinks to the values a deployment sets that the
stylesheet cannot.
`kind: 'segments'` is named by meaning rather than widget — a code read
off one screen and typed into another. The emailed pin and the device
user code are the same field now; they used to describe it separately,
in different files.
### Adding things, after
```ts
providers: {
code: CodeProvider({ ... }),
discord: DiscordProvider({ clientID, clientSecret }), // icon and name included
password: PasswordProvider(PasswordUI({ ... })), // in the design, because it has none of its own
}
```
Neither touches CSS. Neither touches the renderer.
### Notes
- **No Tailwind.** I floated it and then dropped it: `packages/auth`
deploys straight from `src/` both ways (`wrangler.jsonc` points `main`
at `src/index.ts`; `server.ts` runs the same handler under Bun), so
adding a CSS toolchain would fight both. The stringly-typed
`data-component` problem is solved by the typed components instead —
nobody adding a screen writes one. The stylesheet grew ~95 lines for the
new primitives and that is the last CSS this change needs.
- **Design tokens stay independent** of the website rather than shared,
since auth is a separate Worker on a separate hostname with its own
release cadence. The brand values are copied, with a comment naming the
site as the source. Easy to reverse if you'd rather couple them.
- `src/provider/oauth2.ts` has a pre-existing `TS2578: Unused
'@ts-expect-error'` on `dev`. Untouched — verified it fails the same way
without this branch.
### Verification
- `packages/auth`: 66/66 pass. The 27 device tests are **unmodified**
and still pass, which is the behaviour argument — statuses, cookies and
the confirmation step are unchanged.
- `apps/auth`: 23/23 pass.
- `tsc --noEmit` clean on both, apart from the pre-existing error above.
- Every screen rendered and eyeballed in a browser.
Net −757 lines.
|
||
|
Wanjohi
|
cfb8ec26a0 |
feat(machine): mint a public name, and stop routing on the id
A host's hostname was its primary key. That worked and disclosed three things it should not have: ids here are monotonic, so an id in a hostname tells anyone who reads a URL roughly when that machine was registered and where it falls among its owner's others; an id is the primary key, so a name that had to change could only change by re-registering the machine, which is changing its identity to fix its name; and the hostname is also the OAuth audience and the cookie scope, so the id travelled into redirect URLs and browser history. Machines now carry a minted name -- two words and four digits, unique across the fleet, DNS-safe by construction, which an id was not. The words are a curated list rather than a dictionary, because every pair is shown to strangers. Names the fleet's own infrastructure answers on are refused at mint time: one minted onto the edge's own label would take the published key path away from every host at once. Minting retries on the unique index rather than checking first, because two registrations in the same instant both read "free" and both write. Only a name collision retries; a duplicate id or secret means something a new name cannot fix. The migration adds the column in three steps. Generated as a NOT NULL column it fails outright against a populated table, and a default would be worse: every row would share one value on a routing key. |
||
|
Wanjohi
|
eedb143b46 |
refactor(auth): describe sign-in screens as data, draw them in one place
Providers no longer return a `Response`. Each one says what it needs from the person — an address, a pin, a yes-or-no — as a `Screen`, and a single `Renderer` decides how that is drawn. The old arrangement made every provider a small web framework. It had to know about markup, about the stylesheet's attribute names, about how a page is assembled, so each grew its own callback signature and its own copy of `new Response(jsx.toString())`. Three consequences, all of them visible in the tree before this change: - The device flow never got a design at all. Its two pages were built by concatenating HTML strings, with an inline `style` on the user code, and six of its replies were `text/plain` — unstyled black-on-white in the middle of signing in, which is also what a person got when their sign-in cookie expired. - The password screens were drifting. They were written against attribute names the stylesheet no longer had, and nobody noticed because password sign-in is not switched on. They are deleted here rather than repaired; the flow is now six screen descriptions and no markup. - A provider could not be named or marked without editing the library. The brand marks and display names were two hardcoded records inside the code that drew the chooser, so anything missing from them rendered as its own lowercase identifier with no icon. Providers now declare `display` themselves, and the chooser is built from what they say. Also removes the theme global. It was `globalThis`, with a comment conceding as much, which made every component depend on something invisible at the call site — untestable in isolation, and shared mutable state on a runtime that keeps one module instance across requests. The theme is now a closure argument, and the same change shrinks `Theme` to the handful of values a deployment sets that the stylesheet cannot. Adding a screen now touches no CSS, and swapping the presentation layer means implementing one method. The code flow's tests demonstrate the second: they render screens as JSON. Behaviour is unchanged. Status codes, cookies and the confirmation step are the same, which the device tests cover unmodified. |
||
|
Wanjohi
|
c0ecbcb47e |
fix(auth): make the sign-in screen readable, and draw it in the product's design language (#338)
## Why Two separate faults on the same screen, found while trying to sign in for the first time. **The email field was unreadable.** `[data-component='input']` computed its own background one step lighter than the page and never set `color`. Form controls do not inherit it, so the text someone typed was the UA default — black glyphs over a near-black field. There was no `color-scheme` either, so the browser rendered the control in light appearance to begin with. That rule was the only `input` selector in the stylesheet. **The screen was still the upstream template's** — its font, its accent, its logo. `issuer()` takes a `theme` and calls `setTheme`, but nothing had passed one since `packages/auth` became a vendored fork. ## What changed The theme is restored and the page is redrawn in the design language the rest of the product uses: black, a neutral grey ramp, one brand accent, Mona Sans for the display line and Geist for anything read or typed, two dashed bands closing into a box on a wide screen, a dashed vertical either side of the column, the wordmark, and the line of copy the product opens with. **Dark only.** The scheme that let one theme serve both light and dark derived every colour from the background's lightness through `oklch(from ...)`, and that derivation is exactly what left the field's text the colour of its own background. Values are now stated, not computed. The brand colour appears in two places: the wordmark, and the field's border on focus. ## Evidence Measured against a render of the same design built from its own source, at 1280×900: | | reference | this | |---|---|---| | band rules | y `79`, `820` | y `79`, `820` | | column rules | x `106`, `1172` | x `106`, `1172` | | wordmark box | x `505–774`, h `45` | x `505–774`, h `45` | | heading span | `66px` | `66px` | | button height | `60px` | `60px` | Both screens (email and code) and a 390px viewport were rendered and looked at, not just diffed. `bun test packages/auth` → 66 pass, 0 fail. Typecheck, `oxfmt` and `oxlint` clean. ## What this does not verify - **Nothing has been signed in with.** The screens were rendered directly from `CodeUI`; no code has been minted, mailed or redeemed through this page. That is the next thing, and it happens on `auth.nestri.io` after this merges. - **Fonts are a new third-party runtime dependency on the sign-in path.** Mona Sans and Geist come from the jsdelivr Fontsource CDN, because self-hosted font packages need a bundler and nothing preprocesses this page. If jsdelivr is unreachable the page falls back to `system-ui` and stays usable, but serving them from our own origin is probably the right end state. - **Only Chromium was used.** The autofill rules are `-webkit-` prefixed and were reasoned about, not observed; no Firefox or Safari render was taken. - **`docs/deploy.md` is stale** and not touched here. It still says Cloudflare Workers is "what production and sandbox are today"; production is long-lived processes on a VM behind a tunnel. Worth a separate change.prod-c0ecbcb |
||
|
Wanjohi
|
aecae0c69e |
style(auth): tighten the sign-in field, and stop calling the runtime a worker
The email field sat a little taller than the action below it. Two comments also described this page as rendering inside a worker, which is not how the control plane runs; what they were reaching for is that nothing preprocesses this file, so the design tokens are written out longhand. |
||
|
Wanjohi
|
74391714d0 |
feat(auth): draw the sign-in screen in the product's design language
The screen was still the upstream template's: its font, its accent, its logo, and a theme that tried to serve a light and a dark scheme from one set of colours by deriving each one from the background's lightness. That derivation is replaced with stated values, and the page is dark only. Black, a neutral grey ramp, one brand accent, Mona Sans for the display line and Geist for anything read or typed; two dashed bands closing into a box on a wide screen, a dashed vertical either side of the column, the wordmark, and the line of copy the rest of the product opens with. The email field keeps the fix from the previous commit and takes the brand colour on focus, which is the only place it appears besides the wordmark. Measured against a render of the same design built from its own source: the band rules, the column rules, the wordmark box and the button height land on identical pixels. |
||
|
Wanjohi
|
4ed36e0d38 |
fix(auth): give the sign-in input a text colour, and restore the theme
The email field computed its own background one step lighter than the page and never set `color`. Form controls do not inherit it, so the text someone typed was the UA default — black, over a near-black field. There was no `color-scheme` either, so the browser rendered the control in light appearance to begin with. The theme was a second, separate loss: `issuer()` still takes one and calls `setTheme`, but nothing had passed one since `packages/auth` became a vendored fork, so every sign-in rendered as OpenAuth — its font, its periwinkle, its logo. Restored from the pre-fork config, with the logo and favicon repointed because both URLs it carried now 404 and a broken `logo` renders a broken image rather than falling back. |
||
|
Wanjohi
|
155d487525 |
ci: a merge to prod releases the control plane (#336)
The first promotion to `prod` in this repository. Two commits, both CI — the application code is byte-identical to what is already running, so what this merge actually tests is the release path itself. **What it adds** `release-prod.yml`: tests against a real Postgres, both servers executed, the bind default asserted, then three binaries published as an immutable `prod-<sha>` release. A failure anywhere means no release exists, so the machine keeps serving what it has. `nestri-migrate`: a compiled migrator with the migrations baked in, because the deploy runs migrations before it swaps a release into place and had nothing to run. It reimplements drizzle's bookkeeping — same table, same sha256, same high-water mark — because this database was first migrated by drizzle-kit and a disagreement between the two means a migration applied twice. All fourteen hashes were verified by hand against the live database, and CI now asserts the agreement on every build. **Verified before opening this** - Nothing pending against the deployed schema; fourteen rows unchanged. - A scratch database migrated from empty to the same 22 tables and the same high-water mark, idempotent on a second run. - Exit 2 with no `DATABASE_URL`, rather than quietly defaulting to localhost. - The workflow itself, green on `dev` via `workflow_dispatch`, publishing nothing. **What is still untested**, and what merging this will test: migrations run *through the deploy*, and the deploy refusing to swap when they fail.prod-155d487 |
||
|
Wanjohi
|
7859c8e13a |
ci: restart the issuer before the API
Units come back in the order their artefacts appear in the manifest, and the API reaches the issuer over AUTH_INTERNAL_URL -- so api-then-auth means the API spends a moment talking to a service that is restarting. |
||
|
Wanjohi
|
4668c98785 |
ci: a merge to prod releases the control plane, migrator included
`release-prod.yml` for api and auth, mirroring the edge's. The gate is not "you may not merge" — it is that merging does not deploy: tests run against a real Postgres, both binaries are executed, and a failure anywhere means no release exists, so the machine keeps serving what it has. The new piece is `nestri-migrate`, because the deploy runs migrations *before* it swaps a release into place and had nothing to run. `drizzle-kit migrate` reads the migrations folder at runtime, which is right on a laptop and wrong on a server: the deploy ships flat, checksummed files into bin/, and a migrator that needs a directory beside it can be pointed at the wrong directory. So the folder is baked into the binary — generated on every build rather than committed, so it cannot drift — and the artefact's checksum then covers every statement it will run. It reimplements drizzle's bookkeeping in thirty lines of SQL rather than calling into `db.dialect.migrate`: same table, same schema, same sha256 over the whole file, same high-water-mark comparison. That equivalence is the one thing that must not rot, because the production database was first migrated by drizzle-kit and a disagreement means a migration applied twice. Two checks hold it: CI applies the migrations with drizzle-kit and then asserts the embedded set reports nothing pending, and the hashes were verified by hand against the live database — all fourteen match to the byte. Proven before shipping, against the real database: nothing pending on the deployed schema, 14 unchanged rows, and a scratch database migrated from empty to the same 22 tables and the same high-water mark, idempotent on a second run. Refuses with exit 2 when DATABASE_URL is absent rather than defaulting to localhost, which would be a migrator reporting success having migrated nothing. setup-bun is pinned to a commit and not to `v2`. A moving major tag is fine everywhere else in this repository; this workflow is the only thing between a merge and a process serving users, and there is no first-party bun action to prefer instead. |
||
|
Wanjohi
|
b00f1064ae |
fix(api,auth)!: bind loopback by default, and let a container ask for more
Both servers bound `0.0.0.0`. That was harmless while the only deployment was docker-compose.yml, which publishes these ports on 127.0.0.1 and makes the container's own bind irrelevant. As ordinary processes on a rented machine there is no such wrapper, and `0.0.0.0` is a listener on the internet — in front of an issuer that sets cookies without `Secure` and mints sign-in codes, because it expects something else to be terminating TLS. So the default is `127.0.0.1` and `HOST` is there for the deployment that genuinely needs every interface. compose now sets `HOST: 0.0.0.0` explicitly, which is not a workaround: inside a container, binding loopback is what would make the published port unreachable. The right answer differs by deployment, which is why it is a variable rather than a constant. Both now log the address they bound, not the one they hoped for. BREAKING CHANGE: api and auth no longer listen on every interface by default. A deployment that relied on that must set HOST=0.0.0.0. |
||
|
Wanjohi
|
e3c9416170 |
docs(dns): the origin is a tunnel, and there is nothing to cut over from
`After the move off Workers` described proxied `A` records to the host and an ordering to avoid a gap while Worker routes were removed. Both are wrong. There is no `A` record because there is no address to publish: each name is a proxied CNAME to the tunnel, and the machine's address appears nowhere. And there is no ordering problem because `api.nestri.io` and `auth.nestri.io` have no DNS records and do not resolve — only the sandbox pair was ever deployed, so these names are created for the first time with no window to protect. The `nestri.link` section promised an origin certificate as the one key on our host. There is no key on our host at all. Measured 2026-09-16: a public HTTPS request served correctly while nothing listened on 443 or 8443 and both origins spoke plain HTTP on loopback; the certificate was Cloudflare's, covering the apex and one level, and a two-label name was refused at the handshake with no certificate presented. |
||
|
|
15ad60bf49 |
feat: nescapture pacing, improvements and deps update (#334)
Make nescapture better with proper queue family checks, FPS limiting, semaphore usage and other.. also updated deps like pollster and pixelforge. <!-- greptile_comment --> <!-- greptile_summary --> <h2><a href="https://app.greptile.com/api/retrigger?id=64083904"><picture><source media="(prefers-color-scheme: dark)" srcset="https://greptile-static-assets.s3.amazonaws.com/badges/RetriggerDark.svg?v=1"><source media="(prefers-color-scheme: light)" srcset="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1"><img alt="Retrigger" src="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1" align="right"></picture></a>Confidence Score: 5/5</h2> The final review contains no accepted findings, so the PR appears safe to merge. <h3>Summary</h3> - This PR reworks `nescapture` frame capture around a four-slot DMA-BUF ring, tracks presentation queue families, adds semaphore-based capture/present ordering, introduces capture frame-rate pacing, carries presentation timestamps through encoding, and updates Vulkan-related dependencies. <sub>Reviews (1) · Last reviewed commit: ["feat: Update deps and remove deprecated ..."](https://github.com/nestrilabs/nestri/commit/79e21f4aac6f5d15679072bf576ed8cb182805c3)</sub> <!-- /greptile_comment --> --------- Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
8246aa5538 |
feat: resident guest init (#333)
Get this thing going.. <!-- greptile_comment --> <!-- greptile_summary --> <h2><a href="https://app.greptile.com/api/retrigger?id=63134761"><picture><source media="(prefers-color-scheme: dark)" srcset="https://greptile-static-assets.s3.amazonaws.com/badges/RetriggerDark.svg?v=1"><source media="(prefers-color-scheme: light)" srcset="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1"><img alt="Retrigger" src="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=1" align="right"></picture></a>Confidence Score: 5/5</h2> The PR appears safe to merge; all previous findings are resolved and the latest readiness change introduces no established actionable regression. <h3>Summary</h3> - Establishes required guest filesystems, runtime directories, device permissions, and service processes. - Reports initialization and service deaths over the lifecycle channel. - Supports launch, restart, and shutdown commands for a resident guest. - Separates service and workload identities and configures per-launch runtime environments. - Removes the currently inactive nescope screenshot option and makes capture-chain verification fail explicitly when compositor readback is unavailable. - Reworks the guest image around `nesinit` as PID 1 without a distribution service manager. <h3>Diagram</h3> ```mermaid sequenceDiagram participant Host participant Init as nesinit participant FS as Guest filesystems participant Services as Service stack participant Workload Init->>Host: Ready(protocol version) Host->>Init: Boot(mount descriptors) Init->>FS: Establish and mount shares Init->>Services: Spawn services in order Services-->>Init: Required sockets ready Init->>Host: Initialized(service names) Host->>Init: Launch(id, exec, on_exit) Init->>Workload: Spawn with isolated UID/runtime Init->>Host: Started(id) Workload-->>Init: Exit status Init->>Host: WorkloadExited(id, status) Host->>Init: Launch / Restart / Shutdown ``` <sub>Reviews (4) · Last reviewed commit: ["fix(nesinit): readiness is a socket that..."](https://github.com/nestrilabs/nestri/commit/731d34df9df30463f67963363424cb9487e89196)</sub> <!-- /greptile_comment --> --------- Co-authored-by: DatCaptainHorse <DatCaptainHorse@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Wanjohi
|
ec8b13d0c9 |
fix: A host-only session cookie, and a column for where a host actually is (#331)
Closes #330. Two gaps that block a reverse proxy sitting in front of user-owned hardware and authenticating browsers on its behalf, plus one thing found on the way that is worse than either. ## 1. `machine` records where a host is `machine` said who owns a host, which team it belongs to and when it was last seen, and nothing about how to reach it — so a request arriving for a machine could be authorised perfectly and then have nowhere to go. `endpoint_id` is **reported, never assigned**: a host holds the secret half of that identity and is the only thing that can know the public half first, so it rides on the beat it already sends as itself. Migration `0013`. - **Nullable**, because "has never reported one" is a real state that every host registered before today is in. Null reads as *not reachable yet*; a default would read as an address and route somewhere wrong. - **Unique**, because an endpoint id belongs to one host. Two rows claiming the same one would send a request addressed to one machine to another machine's agent, which is the one mistake here the authorisation in front of it cannot catch. - **Omitting the field leaves the stored value alone.** An agent that does not say where it is has not moved, and an absent field must never read as "nowhere" — that would take every host shipped before this field off the map on its next beat. There is a test for exactly that. ## 2. The session cookie, and why it is not shaped the way the issue asked The issue asked for a `__Host-nestri-session` set on sign-in carrying the access token unwrapped. Building it turned up two reasons that cannot work, and item 3 of the issue is the reason why: 1. **There is nowhere to set it.** The sign-in UI is served by the issuer on its own hostname, and the web client has no auth code at all. A `__Host-` cookie set at sign-in is host-only *to the issuer* — a host that does not need one. 2. **The value would be a control-plane credential** sitting on hardware the control plane does not run. Item 3 said the hand-off had to be settled from both ends and did not pick a shape. It is settled now, in the way the issue's own two rules point at: **the proxy is an ordinary public OAuth client, one per hostname**, and what crosses in the URL is an **authorization code** — single-use, sixty seconds long, redeemed exactly once by the store that already exists here, and exchanged over a back channel. A code in an access log is worthless by the time anybody reads it, which is the hazard the issue named. No credential in a URL, and no caller-supplied return address. What this repo owes that flow is one rule, and it is the whole of the change to the issuer: **a client id that is a single hostname under the host zone, whose `redirect_uri` is `https` and that same hostname at one reserved path, is an allowed client.** Everything else keeps the behaviour it had. Making the client id the hostname is load-bearing rather than tidy. A token's `aud` is its client id, so the session that comes back is **bound to the host it will live on**, with no change to how tokens are minted — and it is not a credential on any other host, nor a control-plane credential at all. That is a better answer than "the access token, exactly", and it costs nothing. ## 3. `/authorize` was an open redirector Found while testing the above. A refused client's `redirect_uri` was still used to deliver the refusal — and the check that approves that URI is the one that just failed: ``` GET /authorize?client_id=web&redirect_uri=https://somewhere.example/callback -> 302 https://somewhere.example/callback?error=unauthorized_client ``` No sign-in required, on the hostname people are asked to type a password into. A refusal is now a page here. This is in `packages/auth` and is independent of everything above; it is in this PR because the flow above is built on that path and shipping one without the other would have been odd. ## Tests Every case below fails against the unmodified code and passes after. `379 pass, 0 fail` across `packages` and `apps`. - `apps/auth/test/allow.test.ts` — the allowed case; a code is never sent anywhere but the client id; only the reserved path; `https` only; one label, because `a.b.zone` is not a host id; another zone does not get in by using the path; the three existing rules unchanged; and the refusal is a page rather than a redirect. - `packages/core/src/machine/machine.test.ts` — reported and read back, a silent beat leaves it alone, four malformed ids refused, two machines cannot claim one id. - `apps/api/test/heartbeat.test.ts` — a beat with no body is still a beat, a host cannot report where *another* host is, a malformed id is a `400`. ## What this does not verify - **No end-to-end run against a real browser.** The flow is exercised from the proxy's side and from this side separately; nothing has driven a browser through sign-in and out the other end. - **The token's shape is asserted to the written contract, not captured from a running issuer.** If minting changes, these tests pass and a signed-in person is redirected to sign in again. - **`machine_endpoint_id_unique` is not exercised under concurrency.** Two hosts reporting the same id in the same instant is a database-level race that the tests assert the *outcome* of, sequentially. - **Nothing here makes a hand-off silent.** The issuer keeps no session of its own, so a second hostname asks for an email code again. That is not a regression — nothing anywhere is silent today — but it is the next piece of work, and it is what the issue's item 1 will eventually be, on the issuer's own hostname. - Pre-existing `tsc` errors remain in `apps/api/app/utils/{hook,validator}.ts` and `packages/core/src/session/session.test.ts`; none is touched here. <!-- greptile_comment --> <h3>Greptile Summary</h3> This PR adds a durable, host-reported endpoint identity and permits tightly constrained OAuth callbacks for individual hostnames. It also prevents rejected OAuth clients from controlling the error redirect. - Adds a nullable, unique machine endpoint ID with migration and heartbeat reporting. - Preserves endpoint IDs when legacy agents send bodyless heartbeats. - Converts duplicate endpoint claims into the API’s stable HTTP 409 conflict response. - Allows HTTPS authorization callbacks only at the reserved path on the matching single-label host. - Returns unauthorized-client failures locally instead of redirecting to an unapproved URI. <h3>Confidence Score: 5/5</h3> The PR appears safe to merge; the previously reported endpoint-conflict failure is now translated and tested as a stable HTTP 409 response. No actionable new failure remains. The prior endpoint-conflict finding is fully fixed by translating PostgreSQL uniqueness failures into `already_exists`, which the API maps to 409, with both domain-level and route-level coverage. <h3>Important Files Changed</h3> | Filename | Overview | |----------|----------| | apps/auth/src/index.ts | Adds strict host callback authorization while retaining the existing same-domain and local-development rules. | | packages/auth/src/issuer.ts | Prevents unauthorized clients from using their rejected redirect URI as an open-redirect destination. | | packages/core/src/machine/index.ts | Persists optional endpoint reports and translates duplicate endpoint claims into a stable domain conflict. | | apps/api/app/routes/machine.ts | Extends machine heartbeats with optional endpoint reporting and documents the 409 response. | | packages/core/migrations/0013_machine_endpoint_id.sql | Adds the nullable endpoint column and database-enforced uniqueness invariant. | <h3>Sequence Diagram</h3> ```mermaid sequenceDiagram participant B as Browser participant A as Auth issuer participant P as Host proxy participant H as Machine participant D as Database H->>A: Authenticated heartbeat with optional endpointId A->>D: Update lastSeen and, when supplied, endpointId D-->>A: Stored or unique conflict A-->>H: 200 or typed 409 B->>A: /authorize for host.nestri.link A->>A: Validate matching HTTPS reserved callback A-->>B: Provider flow B->>A: Complete authentication A-->>P: Short-lived authorization code P->>A: Exchange code A-->>P: Host-audience session tokens ``` <sub>Reviews (2): Last reviewed commit: ["fix(machine): a taken endpoint id is a c..."](https://github.com/nestrilabs/nestri/commit/51d25f3e8ce48d4eadef868736334d5dfbd5c05c) | [Re-trigger Greptile](https://app.greptile.com/api/retrigger?id=61026693)</sub> <details><summary><h4>Context used (5)</h4></summary> - Knowledge Base — [API HTTP composition and authorization](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/api-http-and-auth.md) - Knowledge Base — [Core domain and persistence](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/core-domain-data.md) - Knowledge Base — [Machines, pairing, and access tokens](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/core-machines-and-pairing.md) - Knowledge Base — [Authentication platform](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/auth-platform.md) - Knowledge Base — [Auth providers, sessions, and storage](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/auth-providers-and-storage.md) </details> <!-- /greptile_comment --> |
||
|
Wanjohi
|
51d25f3e8c |
fix(machine): a taken endpoint id is a conflict, not a server fault
A host reporting an endpoint id another machine already holds hit the unique index, and the raw refusal reached the global handler as a 500 -- telling a host its beat broke the server rather than that the id is taken. It is now the 409 every other conflict here gives, and the route documents it. Checked-then-written would be worse rather than better: two hosts reporting the same id in the same instant both read "nobody holds it" and both write, which is precisely what the index is for. The read would add a query and remove nothing. Before: expect(res.status).toBe(409) Received: 500 |
||
|
Wanjohi
|
49ae45624e |
fix(auth): a host may receive a code at its own name, and a refusal is not a redirect
Two changes to who may start a flow here, and where a refusal is delivered. A host reached at its own hostname sits on a different registrable domain from this issuer, deliberately: that is what stops a cookie set there from ever reaching this one. The default rule allows a redirect back to whatever hostname the request arrived on, so it refused exactly the case the separation created. Which is a real problem rather than a theoretical one, because a session cookie without a Domain attribute is host-only, so a browser arriving at one of those hostnames for the first time carries no cookie whether or not it is signed in, and sending it here to sign in again changes nothing. So a client id that is a single hostname under that zone, whose redirect_uri is https and that same hostname at one reserved path, is allowed. Making the client id the hostname is the load-bearing part: a token's audience is its client id, so the session that comes back is bound to the host it will live on and is not a credential anywhere else. Separately, and worth its own paragraph: a refused client's redirect_uri was still used to report the refusal. The check that approves that URI is the one that just failed, so /authorize was an open redirector to anywhere at all -- no sign-in required, on the hostname people are asked to type a password into. It is now a page here. Before: GET /authorize?client_id=web&redirect_uri=https://somewhere.example/callback -> 302 https://somewhere.example/callback?error=unauthorized_client |
||
|
Wanjohi
|
6603383ad1 |
feat(machine): record where a host can be reached, as the host reports it
The machine table said who owns a host, which team it belongs to and when it was last seen, and nothing about how to reach it. Anything standing in front of a host and authenticating browsers on its behalf could therefore authorise a request perfectly and then have nowhere to send it. Reported, never assigned. A host holds the secret half of this identity and is the only thing that can know the public half first, so it rides on the beat it already sends as itself. Omitting the field leaves the stored value alone -- an agent that does not mention where it is has not moved, and an absent field must never read as "nowhere", which would take every host shipped before this field off the map on its next beat. Nullable, because "has never reported one" is a real state that every host registered before today is in. Unique, because an endpoint id belongs to one host: two rows claiming the same one would send a request addressed to one machine to another machine's agent, which is the one mistake here that the authorisation in front of it cannot catch. |
||
|
Wanjohi
|
0e94620808 |
feat(nesinit): carry the session's address out of the guest (#328)
## What was missing
`neshub` serves the session's address on a socket, and its own flag has
always
said how that address gets out:
> *"neshub listens; nesinit dials and carries the ticket to the host,
because
> the person who needs it is outside this VM and stdout here is a log
file
> inside one."*
Nothing dialled it. `grep -rn ticket apps/nesinit/src` returned nothing
at all,
so the address never left the guest — and the one lifecycle message for
it,
`Ticket`, had no sender.
This adds the carrier.
## Three decisions worth reading
**Polled, not read once.** An address is not a value, it is the best
answer so
far: an endpoint discovers more ways to reach it after it binds — a
local one
immediately, a relayed one seconds later. Reading once means whoever
asked first
decides, and the first answer is the one that works on a local network
and fails
from anywhere else. Only a *changed* answer is forwarded, so an
unchanged one
costs nothing.
**Dials, does not listen.** The opposite of the payload relay next door,
and
deliberately so. There the guest listens because the workload starts
later; here
the server is the long-lived one. Dialling also makes "not bound yet" an
error to
retry rather than a connection to wait for without knowing whether it is
coming
— which is the ordinary case at boot, since this starts before the
server does.
**The address is never logged.** It is a capability to reach the
session, and a
log inside the guest is the one place it has no reason to be. The log
line says
whether it is the first one and nothing else.
## Failing first
The carrier's tests fail against unmodified code by not compiling:
`ticket.rs`
does not exist and `session::run` takes three arguments. Said plainly
rather
than manufactured. The behavioural gap is better shown as the `grep`
above —
nothing in the guest ever sent a `Ticket`, so the message had one end.
`nesinit`: **35 tests passing**, up from 27. Four on the carrier itself
(an address arrives; a better one replaces it; a socket that is not
there yet is
waited out rather than failed; an empty answer is not an address) and
two on the
session (an address reaches the caller as `Ticket`; a carrier that stops
does
not end the session).
## Verified in a real guest
Built static for musl, run as PID 1 in a real microVM under a real VMM
with a
real vsock. It dialled out, completed the handshake at version 2, took a
boot
descriptor, started its workload, read the address that workload
published and
sent it up the channel — twice, the second time because a better one
appeared.
The caller saw `nestri:local-only` and then `nestri:with-relays`.
Guest boot to init was **310 ms**.
## Review round (
|
