mirror of
https://github.com/nestriness/nestri.git
synced 2026-11-03 22:22:39 +02:00
b296918ab4e71b6d5a5019f21800cc838efb8bea
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Wanjohi
|
b296918ab4 |
feat(api): record what a host says it is running
A host agent already sends a full inventory snapshot on a cadence, and nothing served the endpoint it sends it to — so every one of those calls answered 404. It fails quietly by design, because a dropped snapshot is meant to be corrected by the next one, which is exactly why nobody noticed: the only symptom is a line in the agent's own log. Kept separate from the heartbeat because the two have different loss tolerance. A dropped beat moves a host towards offline and unplaces it; a dropped snapshot costs nothing until the next one arrives. Folding them together would let a malformed inventory field make a healthy host look dead. Three rules decide what a snapshot may do, and the last two are why this is one core function rather than a loop in the route: - a box we know, that the snapshot names, takes the reported state - a box we know that was running, and that the snapshot omits, is stopped and says so — absence inside a snapshot is information - a box the snapshot names that is not placed on the calling host is never created, only reported back as a divergence The scope is in the `where` clause and not in the agent asking politely about its own boxes: a machine credential is a long-lived secret sitting on hardware in somebody's living room. `pid` and `uptimeS` are accepted and deliberately dropped. A pid is a number in another machine's namespace, and uptime is derivable from a run's start time, which is already stored and already trustworthy. |
||
|
Wanjohi
|
9258c8dfef |
fix(deploy): make bun dev actually start, and sign-in actually work
Six defects found by running the thing rather than reading it. The previous change was verified by bundling, by tests, and by the container images — none of which start a Worker, so every one of these was invisible. **`bun dev` did not start.** It ran one multi-config process, which does not connect a service binding between the workers it loads; the API reported `AUTH [not connected]` and could not verify a token. It is two processes now, which is what the dev registry connects, and the second is backgrounded with the first killed on exit so stopping the pair stops both. **Neither server could bind.** Wrangler resolves `localhost` and takes `::1` first; a host with no IPv6 address on its loopback dies with a bind error from inside the runtime that names neither the app nor the port. `dev.ip` is pinned to `127.0.0.1`, and `inspector_port` is now distinct per app — it is not derived from the port above, so the second server to start died on an address already in use. **The API worker failed to evaluate.** A specifier ending in `.sql` is claimed by the bundler as a module of its own, so the schema file was emitted verbatim beside the bundle and the runtime threw on an export it could not find. The route was reaching past the domain module into the schema to spell a status; it now asks the domain module, which is the rule everywhere else here and happens to also avoid the hazard. **Signing in failed on the second request that touched the database.** A pool is cached per connection string, and on a Worker an I/O object created while handling one request may not be touched while handling another. The first request always succeeded, which is why it went unnoticed — a sign-in is several. The cache is now kept only where a process outlives its requests, which is the case it was added for. **The images named a base that podman will not resolve.** A short name needs a registry; the database service alongside them already spelled one. **Compose pinned container names.** The name is not scoped to the project, so a second checkout got the same three, and `down` in one stopped the other's containers. This is not hypothetical — it stopped a running development database while this was being tested. Verified by signing in end to end against both dev servers: a code requested over HTTP, read from the issuer's log, redeemed, exchanged for tokens, and presented to the API, which resolved it to the account the sign-in had just created. |
||
|
Wanjohi
|
f30a1432f8 |
fix(deploy): require every credential, and give sandbox its own domain
Three things review caught, and one shape correction. **No credential has a default any more.** The compose file shipped `ADMIN_SHARED_SECRET` falling back to a value written in this repository — and that header bypasses token verification entirely, so anyone reading the file could act as an operator against any deployment that had not overridden it. A default is worth less than it looks here: the deployment that never set the variable is exactly the one where the default is public. Every credential now comes from `.env`, and compose refuses to start naming the variable it wanted. That also takes the last literal password out of a tracked file. **The origin ports are on loopback.** Both services speak plain HTTP and mark no cookie `Secure`, because both expect to sit behind something that terminates TLS. Published on every interface they were a way to reach the issuer around that proxy, with sign-in codes and tokens in clear text. **Mail settings are passed through rather than fixed.** The issuer was pinned to printing sign-in codes to its log, and the three delivery settings never reached it — so the documented way to configure mail could not work, and every code and recipient went to the container log instead. Printing codes is now asked for in `.env` like everything else, and with nothing configured the issuer refuses to send rather than logging. **Sandbox becomes a domain rather than a prefix.** `api.sandbox.nestri.io` and `auth.sandbox.nestri.io`, because sandbox holds whatever is not production and that set grows. One certificate for `*.sandbox.nestri.io` then covers all of it, including unpredictable per-pull-request names, and cannot be presented for production's own domain — which the zone-wide wildcard the previous shape leaned on could. Also drops `STEAM_API_KEY`. It was declared in two type definitions and read by nothing: linking an account makes no outbound call that needs it. |
||
|
Wanjohi
|
51ababc900 |
feat(deploy): drop the IaC layer, and make both apps runnable as containers
Moving the issuer's state into Postgres removed the last thing that tied either app to one hosting provider. What was left was a deployment tool describing resources that no longer existed — so this replaces it with `wrangler`, which is what actually deploys a Worker, and adds a second way to run each app that involves no provider at all. Each app now has a `wrangler.jsonc` with an environment per stage, and a `Dockerfile` beside it. The handler is the same one in both cases; what differs is only where its settings come from. Two of them gained a second spelling so that nothing has to branch on the runtime: Postgres arrives as a pooled binding or as `DATABASE_URL`, and the route to the issuer is a service binding or `AUTH_INTERNAL_URL`. That last one is new, and it is a split the binding was already making without saying so. `AUTH_ISSUER_URL` has to be the issuer's public name, because it is compared literally against every token's `iss` claim — but the public name is often not routable from inside a deployment. So the name and the route are two settings now rather than one that cannot be both. DNS moves out of code and into `docs/dns.md`, which lists every hostname and what it is for. Six records that change roughly never did not need a tool, and the table outlives whatever is answering the names — which is the point, since some of them will stop being Workers. The sandbox hostnames are hyphenated rather than nested for the same reason: a certificate covering `*.nestri.io` covers one label and not two, so `api-sandbox.nestri.io` can become an ordinary origin later without a certificate having to be ordered for it first. Also drops `EMAIL_DEV_LOG` from committed configuration into `.dev.vars`, which `wrangler deploy` cannot upload. Printing a live sign-in code to a log should not be one forgotten override away from production. |
||
|
Wanjohi
|
54d5c81edb |
feat(api): hold a run to the attempt that claimed it
The agent side sends a claim token on every write; this side rejected the field outright, so every state report and every ticket publish answered 400. Both bodies now take it. Underneath that, nothing compared a holder. A run was reachable by any caller on the right machine, and a box names exactly one machine — so two attempts polling the same job presented identical credentials and were told apart only by which one's select landed first. That is timing, not a rule, and no caller could be told which case it was in. The row now remembers which attempt holds it. Taking a claim requires there to be no holder; every write after it requires the caller to be the holder. The same state reported by a different attempt is a lost race and not a retry, and is refused whatever the state is - which is the only thing that separates the two 200s from the 409s. The ticket is held to the claim too, for a worse reason than a double start: the client re-reads the address rather than keeping the first, so a ticket written by a losing attempt produces a client that connects, successfully, to a machine running nothing. The holder is never cleared, including on a terminal state, so a settled claim cannot be replayed and a finished run still records which attempt ran it. It is not in what goes out - holding one permits writing to a run, and the owner reading their own session is not the holder. |
||
|
Wanjohi
|
7bdca1240f |
fix(api): say what the library check actually proves
A library entry records the person, not the account the games were synced from, and `POST /library/sync` is not even told which account a list came from. So the ownership check added for session requests asks "has somebody this person linked got this game?" and not "does the account about to play own it?" — for the one Steam account most people have those are the same sentence, and for two they are not. Confirmed rather than reasoned about: a person with two Steam links, a game synced at person level, and a request naming the second account is accepted today. The check stays, because it still turns a box that boots, tries to launch and fails minutes later into an immediate refusal, and it never refuses on account grounds that the data cannot support. What changes is the comment, which claimed the stronger property, and a test that pins the gap so it is found deliberately rather than by surprise. Closing it properly means recording the linked account on a library entry: a column, a sync contract that says which account a list belongs to, a uniqueness rule per account rather than per person, and a backfill with no correct answer for rows already written. That is a decision about what a library is, and inferring it here would be the kind of modelling taken by accident that this branch refuses elsewhere. |
||
|
Wanjohi
|
51dabddbd8 |
fix(api): a run drives the box under it, and needs a game and a claim
Four things the session endpoints did not do, or did wrongly. The box had three states and nothing wrote them. A box read `created` while a run on it was `live`, so every screen showing a person what their hardware is doing was reading a column no code had ever moved. A run reaching `live` now makes its box `running`, and a terminal run stops it: `ended` cleanly, `failed` not, carrying the reason the agent gave. Not every run state maps — a box has no `starting` on purpose, because that transition is synchronous from the agent's side and a state nobody sets is a state that lies. Both writes are one transaction, since "this run is live" and "the box under it is running" are one fact in two tables, and a box stuck `running` with nothing on it has nothing to correct it. `POST /session` accepted any game in the catalog. A run launches as a Steam account that has to own the game, so one outside the caller's library is a box that starts, tries to launch and fails minutes later with nothing to point at; it is now refused up front. Told apart from a game that does not exist rather than hidden, because the catalog is public and "you do not own this" is a sentence a person can act on. The library is a synced copy, so this refuses a game bought since the last sync — that is a staleness bug in the sync, not a reason to start runs that cannot work. Publishing a ticket only refused terminal runs, so a host could publish an address for a run it had never claimed. A ticket is the address of something being brought up, so only `starting` and `live` accept one, and the state is in the write rather than only in the check above it. The two refusals stay separate answers because they are different mistakes: one agent skipped a step, the other has nothing left to reach. The migration that adds the one-active-run index stopped older duplicate runs without clearing the ticket they had published, which is the invariant that same migration exists to establish. It clears it now, verified against a box carrying two unstopped runs. Nine tests, each checked against the unfixed code first. |
||
|
Wanjohi
|
0d8630379b |
fix(api): a box gets one run, and a stopped run keeps no address
Two invariants the session endpoint stated but did not hold. A box runs one thing at a time. `POST /session` read `activeForBox` and refused when something was already running, but the read and the insert are two statements with nothing between them: two requests that both saw "nothing is running" each got a row, and the job poll then handed the host the same box to start twice. Demonstrated at 2 rows and 2 jobs from one box. That is the failure the state claim exists to prevent, one step earlier, and it takes the same answer — a partial unique index on the predicate the read asks about, so the database refuses the second insert. `Session.request` turns that refusal into the same 409 in the same words, so a caller cannot tell which of the two caught it. The migration resolves any existing duplicates before creating the index, keeping each box's newest unstopped run because that is the one a person is waiting on, and stopping the rest rather than deleting them. Separately, a run that reached `ended` or `failed` kept the last ticket it published. Publishing a new one is already refused, so the stale address was both the only ticket a client could read for a dead run and the one nothing was allowed to replace — and a client that polls would dial it. Terminal transitions now clear it, in `setState` as well as in the compare-and-set, so the invariant does not depend on which writer stopped the run. Seven tests, each checked against the unfixed code first. The published descriptions for the ticket field and the read endpoint now say that a stopped run has no address. |
||
|
Wanjohi
|
bbe729e5c7 |
feat(api): the session endpoint, and a claim that only one caller can win
A run of a box had core support and no HTTP surface. This adds both halves of it: a person asks for a run and reads it back, and the host agent the box is placed on is handed the work and reports what happened. The access rule is the point. An agent may only see or touch a run whose box is placed on its own hardware, and that is a `where` clause on every one of the three agent endpoints rather than a check next to them — host credentials are long-lived secrets sitting on hardware in somebody's home, so what one leaking can reach has to be decided by the query. "No such run" and "not your run" are the same refusal, so ids cannot be discovered by reporting states at them. `Session.setState` updated on the id alone, which means two agents polling the same work both succeed and both start the same box. There is one host today, which is exactly why that would have been built wrong and stayed wrong. The state a run is moving out of is now part of the `where` clause, so the database picks the winner; the loser gets a conflict rather than a silent no-op. Three cases that look alike are kept apart: re-reporting a state you already reported changes nothing and is not an error, a transition that does not exist is refused with the run left where it was, and another host reporting anything is forbidden. Asking for a run makes no decision about where it happens — a box already names its hardware, so the run inherits it by join. Placement therefore gets an interface at box creation, where the decision actually is, with the single-host case as its implementation and a deliberate refusal when there is more than one candidate and no policy to choose with. Tests cover the wire shape from both sides, the query scoping, the claim, and the timestamp idempotence a run's billing rests on. |
||
|
Wanjohi
|
cf56aaf04c |
docs: this repo is public, so say what things are, not who decided them
Comments and served API descriptions here had grown references that only make sense to someone with our internal notes: relative paths that escape this tree, filenames and titles of documents nobody outside can open, quoted prose from them, and the name of a component that has no public surface — once in an OpenAPI description, which is published output rather than source. None of it was load-bearing. Every case restates as what the code actually requires, and every rewrite came out shorter: "in the words the host agent reports" for a component name, "republished as addresses are discovered" for a quoted phrase, "a size tier sets vCPU, RAM and the output geometry" for a sentence that had been carrying a path. Internal reasoning is now cited exactly one way, ref(d-NNNN) in a source comment, with the rule that the sentence must still stand if the marker is deleted. CLAUDE.md leads with it, because the previous version of this mistake was made by people who knew the repo was public and it still took ten occurrences to notice, so "be careful" is not a mechanism. Commit messages get the stricter rule and carry no references at all: a comment can be fixed by the next commit and a published message cannot be fixed at all. Git hooks now enforce both halves. The check caught a real one while being written: the CLAUDE.md table spelled out the paths it was prohibiting, which discloses them to exactly the reader it protects against. 138 tests, 0 fail. |
||
|
Wanjohi
|
c3682136f1 |
test(api): hold the heartbeat wire contract from the host's side
`lastSeen` and `intervalSeconds` are the two field names neslet's plane.rs reads out of the reply, and a rename on either side yields a host that beats, parses nothing and reports success. These tests are what make that a contract rather than a coincidence. Also asserts the property the middleware comment claims and nothing checked: wrong machine credentials and no credentials produce *identical* responses, because bad credentials fall through to `public` rather than erroring so that probing cannot reveal which machine ids exist. Comparing the two bodies is the only way that stays true. Two of these tests started out asserting 401 and were wrong, not the code — `machineOnly` sees a public actor either way and forbids. 138 tests, 0 fail. |
||
|
Wanjohi
|
4315510de8 |
feat(api): a host can say it is alive, and is told how often to
Second half of G1's "neslet registers against api.nestri.io and heartbeats".
Registration already worked; there was no heartbeat endpoint at all — grep for
it across apps/api and packages/core returned nothing, and neslet's own
main.rs says the same from its side.
POST /machine/heartbeat, machine credentials only. Two decisions worth stating
because neither is obvious from the diff:
**It returns the interval.** The auth middleware already touches lastSeen on
every authenticated machine request, so an endpoint that only did that would
add an endpoint and no capability. What a host cannot know on its own is how
often the control plane wants to hear from it, so the response carries the
cadence. A fleet whose interval can only change by shipping a new agent is a
fleet whose interval never changes.
**It takes no body.** neslet has a HostSummary ready to send, and week 2 owns
box state reporting. Accepting fields nothing acts on yet would mean a wire
shape we would have to keep, chosen before the thing that consumes it exists.
Online-ness is derived from lastSeen rather than stored: a host that stops
beating goes offline through the passage of time, which is the one mechanism
that cannot itself fail. Three missed beats, not one — a single missed beat is
a lost packet, and treating that as offline would make placement flap.
Also: the machine actor's teamID stops being optional. It was `...(teamId ? {}
: {})` in the middleware, a branch for a state that cannot exist now that
machine.team_id is notNull.
134 tests, 0 fail.
|
||
|
Wanjohi
|
6c1d407985 |
feat(core): a box is a row, a session is the billing unit
Migration 1 of 0048, and the first of the seven weeks — nothing about a live
feed works without these two tables, so it is not a cleanup during them.
box a VM someone owns: an id that is also its DNS label, an editable
label, an owning user, the machine it sits on, a tier and a state.
Owned by a person and placed on a team's hardware, which are two
different relationships, hence both userId and machineId.
session one run of one box by one linked Steam account, and what costs
money. Separate from box because the ticket changes after bind as
addresses are discovered — the vsock contract calls it "a stream,
not one value" — so it is a column a client polls, not a value it
is handed once.
Box states are neslet's own three and no more. `starting` and `stopping` are
the obvious additions and both are omitted because nothing would ever write
them; a failed box is `stopped` with stopClean false, which is how neslet
models it too.
The generated migration would have failed on live rows in three ways, so it
is hand-written and tested against a database seeded at the old schema:
- machine.team_id becomes notNull, and *every existing row is null* because
the old registration path passed null. Personal teams are backfilled for
machine owners first, reusing a team they already own rather than minting
a second, with the owner membership row repaired where missing.
- game_download.host_id becomes a foreign key. It held free-form strings,
so unattributable rows are deleted before the cast — the only destructive
statement here, and a considered loss: it is a progress report neslet
re-derives from disk.
- Team.createPersonal was written and documented in packages/core/CLAUDE.md
as part of the login flow and never actually called, so no user has a
team. ensurePersonal is idempotent and now runs on every login, which is
what backfills accounts the migration does not reach.
Verified on a seeded legacy database: three null-team machines backfilled, an
existing team reused rather than duplicated, a blank display name handled, and
both unattributable download rows dropped while the attributable one survived.
Also fixes two things this work ran into rather than caused:
- Database.client() built a new postgres pool on every call, and use()
called it twice per invocation — pools of ten connections held for a 30s
idle timeout. Invisible in a Worker where requests are short; the suite
crossed 100 connections and Postgres said "sorry, too many clients
already" in whichever file ran last, which reads as a flaky test rather
than a leak. Now one pool per connection string.
- download.test.ts asserted against `hst_…` host ids, which is exactly the
unattributable row the new foreign key exists to refuse.
There is no "no team" any more: PATCH /machine/:id took teamId null to mean
"mine alone" and now requires a team, because the personal team is the one to
name. Its test is updated to the new contract rather than deleted.
113 → 128 tests, 0 fail.
|
||
 WanjohiandClaude Opus 5
|
77d4782c86 |
docs: split CLAUDE.md by scope, and say what this repo is
CLAUDE.md was 1,324 lines and all of it was about the TypeScript half, written before there was another half. Every line of it loaded on every turn regardless of what was being worked on, which is a real cost paid constantly for context that is usually irrelevant. Split by where it applies, so each guide loads when you are in the directory it describes: packages/core/CLAUDE.md 694 domain modules, fn(), actor, errors, auth apps/api/CLAUDE.md 284 routes, registration, error flow docs/alchemy.md 345 stages, bindings, secrets, the CLI CLAUDE.md 72 the repo, both toolchains, two hard rules Nothing was rewritten or dropped — the three files are the original text, verified identical after the split. What the root file now carries is only what is true repo-wide: the layout, the commands, where the detail lives, and the two rules that are not style preferences. One of those is that nothing closed may enter this repo, which is here because it has already been caught once. The README described a streaming platform in four bullets and did not mention that half the repository is Rust that runs inside a virtual machine. It now says what each component does, why a micro-VM rather than a container, what is deliberately absent, and what decides whether a thing is open — data is, capacity is not. It also says plainly that this is mid-rewrite and the docs are behind. Someone arriving at a repo whose documentation does not match its tree should be told that by the README rather than discover it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
WanjohiandClaude Opus 5
|
0143849129 |
feat: bring the control plane up to date
Squashes the current state of the internal working tree onto this history. The two trees had grown apart with no common ancestor, so this is a content sync rather than a merge, and the published history is preserved rather than rewritten — a force-push here would break every existing fork and clone to no benefit. What lands: - Waitlist: API route, core module, and migration 0006 alongside game aliases. - User verification. - CI, oxfmt config, editor settings. - Assorted fixes across the API routes and core modules. The repository's own README, the wordmark and the per-package READMEs are kept from this side; the internal tree had dropped them and they are what a stranger arriving here reads first. The marketing site in the internal tree is deliberately not here. It is a separate product with its own repo and its own licence, and this repo is the open one — a closed component does not belong in it regardless of how convenient the directory looked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Wanjohi
|
a8b9a11de0 | chore: Update Readme | ||
|
Wanjohi
|
3faac3008f | feat: Sync to OSS repo |