3d0dcf3e46
Moves the issuer's state out of Cloudflare KV and into Postgres, and
splits it by whether a record actually needs the guarantees a key-value
store cannot give.
## Why
The issuer kept everything behind one `get`/`set`/`remove`/`scan`
interface — which is what a library that has to run on any provider's
cache can offer. Three of the things kept there could not be served by
it:
- **Authorization codes** must be redeemable once. `get` → validate →
`remove` are separate steps, so two exchanges of one code arriving
together both pass every check, and each answer is a complete session.
- **Refresh tokens** must be spendable once. Reuse detection works by
recording *when* a token was first spent, so the check and the record
have to be the same operation. Split apart, two refreshes both look like
the first — and the reuse that reveals a stolen token is never recorded,
because recording it is the write the second caller overwrites.
- **Signing keys** don't race, but they're the one record whose loss
ends every session at once, and a cache is a place things may be evicted
from.
This is the same argument `device_grant` already made, applied to the
records that had it too.
## What's here
| table | why it exists |
|---|---|
| `authorization_code` | redeeming is one `delete … returning` |
| `refresh_token` | spending is one `update … where time_used is null
returning *`; `subject` indexed, so signing out everywhere is a query
rather than a prefix scan |
| `auth_key` | retired by setting `expired_at`, never deleted, so tokens
stay verifiable through a rotation |
| `auth_kv` | everything left: the rate-limit counters |
`auth_kv` stays deliberately generic. Those counters are written far
more often than read, meaningless within the hour, and allowed to be
approximate — a lost increment costs one guess out of ten. It's the one
place an unmigrated `jsonb` blob is the right answer rather than a
shortcut.
Both credential tables store a **hash and never the credential**, as
`device_grant` does. An authorization code travels in a query string, so
it passes through browser history, referrer headers and any log along
the redirect; a refresh token resumes a session outright.
`packages/auth` gains `keyStore`, `codeStore` and `refreshStore` as
optional issuer inputs, each defaulting to a storage-backed shim so the
library behaves exactly as before when they aren't passed — same shape
as the existing `deviceStore`.
## Portability
`AuthStorage` was the only stateful Cloudflare-proprietary primitive in
the control plane. It's gone, so the remaining CF surface is Hyperdrive
(a pooler over a plain `DATABASE_URL`), Workers and DNS. The self-host
story stays one service.
## Two behaviour changes worth reviewing
1. **A code that fails a check is now spent.** `consume` happens before
the redirect-URI, client and PKCE checks, because the operation that
decides which caller gets the code has to be the one that removes it.
RFC 6749 §4.1.2 asks for this, but it is a change: a code presented
wrongly no longer gets a second try.
2. **`legacySigningKeys` is removed.** It read a pre-ES256 `oauth:key`
prefix and stamped every key it returned with a hardcoded expiry of
2025-01-02 — so everything it produced has been expired for over a year,
and it only ever read from the store being left behind.
## ⚠️ Deploying this signs everyone out
The signing keys and refresh tokens live in the KV namespace this
removes. The issuer will start with a fresh key set, so every existing
access token stops verifying and every refresh token is gone. Worth
timing deliberately, or copying the key rows across first if that isn't
acceptable.
## Testing
- `packages/auth` — 66 pass (the issuer flows run through the
storage-backed shims, so the rewrite is behaviour-preserving on the
default path)
- `packages/core` — 130 pass, including 24 new ones
The concurrency claims are tested rather than asserted: five overlapping
`claim`s of one refresh token yield exactly one `fresh` and four
`reused`; five overlapping `consume`s of one code yield exactly one
non-null. Also covered: `LIKE` wildcard escaping in `scan` (keys are
built from email addresses and caller addresses, so `%` and `_` are not
hypothetical), and that `scan(['a'])` no longer reaches into `['ab']`.
Migration `0011` applies cleanly from empty.
## Not in scope
Pre-existing `tsc` errors — JSX config for `packages/auth/src/ui/*.tsx`,
`subject.ts`, `oauth2.ts`, `session.test.ts` — are untouched and left
for their own PR.
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
This PR moves issuer state from Cloudflare KV into dedicated PostgreSQL
stores and adds atomic operations for authorization-code redemption,
refresh-token claims, and signing-key bootstrap.
- Authorization codes are hashed and consumed with `DELETE ...
RETURNING`.
- Refresh tokens are hashed and claimed with a conditional atomic
update.
- Signing keys are persisted in PostgreSQL with one live key permitted
per kind.
- Remaining approximate rate-limit state uses a generic
PostgreSQL-backed adapter.
- The changes since the previous review preserve each stored key’s
algorithm and make concurrent key bootstrap converge on one live key.
<h3>Confidence Score: 5/5</h3>
The PR appears safe to merge; both findings from the previous review are
resolved and no new actionable failure remains.
Concurrent key bootstrap now converges through a partial unique index
and a post-insert reread, while imported keys retain their stored
algorithms. Both previous threads were manually resolved, and the
current code confirms their underlying issues are fixed.
<h3>Important Files Changed</h3>
| Filename | Overview |
|----------|----------|
| packages/auth/src/issuer.ts | Routes authorization codes, refresh
tokens, signing keys, and invalidation through specialized stores. |
| packages/auth/src/keys.ts | Preserves stored key algorithms and
rereads the store after bootstrap so concurrent issuers converge. |
| packages/core/src/auth/signing-key.ts | Persists key material and
safely ignores a concurrent insertion that already established the live
key. |
| packages/core/src/auth/authorization-code.ts | Implements atomic,
single-use authorization-code consumption. |
| packages/core/src/auth/refresh-token.ts | Implements atomic
refresh-token claims and subject-wide revocation. |
| packages/core/migrations/0011_auth_state_in_postgres.sql | Adds
PostgreSQL tables and constraints for issuer state, including one live
key per kind. |
<h3>Flowchart</h3>
```mermaid
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Issuer[OAuth issuer] --> Codes[Authorization code store]
Issuer --> Refresh[Refresh token store]
Issuer --> Keys[Signing key store]
Issuer --> KV[Generic auth state]
Codes -->|DELETE RETURNING| PG[(PostgreSQL)]
Refresh -->|Conditional claim| PG
Keys -->|Partial unique live-kind index| PG
KV -->|Counters and rate limits| PG
```
<sub>Reviews (2): Last reviewed commit: ["fix(auth): keep one live key
per kind,
a..."](f64f037574)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=60727758)</sub>
<!-- /greptile_comment -->
Run your games on a GPU you don't own — or one you do. Nestri puts an interactive workload in a hardware-accelerated virtual machine and streams it to you over QUIC, at a latency that lets you play rather than watch.
Note
This repository is mid-rewrite, and the documentation is behind the code. The guest-side components arrived recently and their docs are thin. Nothing here is stable yet: expect directories to move and interfaces to change. Proper documentation is on the way — issues and questions are welcome in the meantime, and are genuinely useful for deciding what to write first.
Try it now — nesdoctor
One thing here is finished and runs on its own machine, today:
# Linux and macOS
curl -fsSL https://doctor.nestri.io/install.sh | sh
# Windows
powershell -c "irm https://doctor.nestri.io/install.ps1 | iex"
It tells you whether your machine could host games for other people, and
measures the number that actually decides whether streaming a game feels
right — not your download speed, but how much latency your connection adds
when it is busy. A 500 Mbps uplink that queues for 300 ms under load cannot
carry a game; a 25 Mbps one with fq_codel can. Almost nobody has seen their
own figure.
upstream 35 Mbps
latency, idle floor 56 ms
latency, loaded 185 ms
added under load +129 ms grade F
presentation path x11 · bspwm
eDP-1 1920x1200 @ 60 Hz, 8-bit
Vulkan decode h264, h265
It also reads your display out of its EDID — resolution, refresh, colour depth, HDR transfer functions, BT.2020, chroma — and what your hardware can decode. Those decide what is worth sending over the wire, and we would otherwise be guessing from one panel in one room.
It does not stream a game. It is the piece that has to exist before
anything else can, and most machines will come back CLIENT — which is a real
answer, not a failure.
Downloads one binary, verifies its checksum, runs it, deletes it. Installs
nothing, needs no administrator rights, touches no system directory. Nothing is
uploaded: it prints a link, lists exactly what the link contains, and opens it
only if you press Enter. The scripts those URLs serve are
apps/nesdoctor/install/ in this repository, so you
can read them before you run them.
Source and the full story: apps/nesdoctor.
What is here
Two halves that meet over the network and share very little else, plus one thing that runs on your own machine.
The control plane — TypeScript, on Cloudflare Workers
apps/api |
The public REST API. Identity, teams, machines, games, pairing. |
apps/auth |
A self-hosted OpenAuth issuer — Steam and SSH-key login. |
packages/core |
The domain: every table, every operation, no HTTP. |
packages/auth |
Shared auth types and subjects. |
Postgres for state, Alchemy for infrastructure. See
docs/alchemy.md.
The guest — Rust, inside the box
These run inside a virtual machine, beside the game. None of them talk to the control plane.
apps/nescope |
A headless Wayland compositor for one fullscreen client. A lighter answer to the same problem gamescope solves. |
apps/nescapture |
A Vulkan implicit layer. It captures frames from inside the workload's own process and encodes them on the GPU that drew them — no copy out to the CPU and back. |
apps/neswire |
Audio capture and transport. |
apps/neshub |
One connection out of the box. Muxes video, audio, cursor and input into a single QUIC stream to the client. |
crates/nesprotocol |
The wire types they all share, so no two ends can drift apart silently. |
On your own machine — Rust
apps/nesdoctor |
Whether a machine can host a box, and what its connection and display can really do. The first executable form of our host requirements — until it existed, a host was qualified by a human reading a table. Four dependencies; everything that could be done with the standard library is. |
The hypervisor the guest components run under is nesbox,
a separate repository: a micro-VM with a real GPU in it, using virtio-gpu native
context rather than passthrough, so one card can host several boxes at once.
Why a virtual machine
A container shares the host kernel, which makes strong isolation hard and a GPU harder. A micro-VM boots in about as long, isolates properly, and — with native context — gets close to bare-metal graphics. That choice is what makes "many sandboxes, one GPU" possible instead of one tenant per card.
Getting started
bun install
bun dev # control plane, local Cloudflare runtime
cargo build --workspace # guest components
cargo test --workspace
The guest components expect a Linux host with a Wayland-capable GPU stack, and are not much use on their own yet — they are pieces of a box, and the thing that assembles a box is not open yet.
nesdoctor is the exception and needs none of that:
cargo run --release -p nesdoctor
Status
Working: nesdoctor — released, and the only part a stranger can operate
today. The API, auth, the domain model, and the guest components listed above.
Not here yet: the box lifecycle, storage, the edge, and the client. Some of that will open as it is written; some is deliberately closed. What decides which is whether it handles your data — that half is open on principle — or decides our capacity, which is the part we sell.
Contributing
Early, and the ground moves. The two most useful things you can do right now
cost a minute each: run nesdoctor and send the result, because we have
almost no idea what the machines on the other end of this look like; and tell
us where the documentation failed you. Conventional commits; explain why in
the body.