mirror of
https://github.com/nestriness/nestri.git
synced 2026-09-19 09:15:19 +03:00
feat(api): the session endpoint, and a claim that only one caller can win (#317)
## What this is
`POST /session`, `GET /session/:id`, and the three endpoints the host
agent
calls: `GET /machine/jobs`, `POST /session/:id/state`, `POST
/session/:id/ticket`.
Core had `Session` and `Box` and no HTTP surface at all; this adds the
surface
and the two things core was missing to support it safely.
### The access rule
An agent may only see or touch a run whose box is placed on its own
hardware,
and that is a `where` clause on all three agent endpoints — not a check
sitting
next to the query, and not the agent asking politely for its own work.
Host
credentials are long-lived secrets on hardware in somebody's living
room, so
the blast radius of one leaking is decided by the join and nowhere else.
`Session.listJobsForMachine`, `forMachine`, `compareAndSetState` and
`publishTicket` all scope to the machine inside the statement. The
compare-and-set is scoped there too, and not only by the read above it:
a
caller checking first and the write being scoped are different
properties, and
only one of them survives somebody adding a second caller.
"No such run" and "not your run" are the same 403 with the same body, so
reporting states at ids cannot be used to discover which ids exist.
### The claim
`Session.setState` updated on the id alone. Two agents polling the same
run
both succeeded and both started the same box. There is one host today,
which
is exactly why that would have been built wrong and stayed wrong.
`Session.compareAndSetState` puts the state being moved *out of* into
the
`where` clause, so the database picks the winner and the loser matches
zero
rows. `Session.transition` layers the classification on top and returns
one of
five outcomes, which the route maps:
| Case | Answer |
|---|---|
| same host re-reports a state it already reported | `200`, nothing
changes |
| a different host reports anything | `403` |
| a transition not in the table | `409`, row does not move |
| a legal transition something else won first | `409` |
| it happened | `200` |
`setState` is left exactly as it was — it is the unscoped primitive the
core
tests already pin, and the new path is additive.
### Placement
`POST /session` makes no placement decision. A box names its hardware,
so a
run inherits it by join, and the request body is `.strict()` so that
naming a
machine there is a validation error rather than a field quietly ignored.
The interface went where boxes are actually placed: `Placement.Placer`
in
`packages/core/src/box/placement.ts`, with `Placement.onlyHost` as the
implementation and `Box.createPlaced` as the one caller. A real
scheduler
replaces that file and nothing else.
## The test failing first
Tests were written and run against unmodified code. Verbatim, trimmed to
the
result lines (the full log is stack traces for the same failures):
```
(fail) POST /session > a request creates the job, in the envelope both ends read [199.00ms]
(fail) POST /session > creating a session makes no placement decision [55.00ms]
(fail) POST /session > a box somebody else owns is not there to run [106.00ms]
(fail) POST /session > a box already running refuses a second run rather than picking one [51.00ms]
(fail) POST /session > you can only play as an account you have linked [85.00ms]
(fail) POST /session > a host cannot ask for a session on its owner’s behalf [54.00ms]
(fail) POST /session > requesting a session requires a signed-in person [1.00ms]
(fail) GET /session/:id > the owner reads their own run, ticket and all [47.00ms]
(fail) GET /session/:id > somebody else’s run is not visible, and neither is its absence [97.00ms]
(fail) GET /session/:id > reading a run requires a signed-in person [1.00ms]
(fail) GET /machine/jobs > a host is handed the work for its own boxes, with the kind on the wire [50.00ms]
(fail) GET /machine/jobs > a host never sees work for a box on other hardware [84.00ms]
(fail) GET /machine/jobs > bad credentials are indistinguishable from none [41.00ms]
(fail) GET /machine/jobs > a person cannot poll for jobs [44.00ms]
(fail) POST /session/:id/state > the claim moves the row, and the job stops being offered [41.00ms]
(fail) POST /session/:id/state > the same host re-reporting a state it already reported is fine [54.00ms]
(fail) POST /session/:id/state > a different host reporting anything is refused, and learns nothing [99.00ms]
(fail) POST /session/:id/state > a transition that is not allowed is a conflict, and the row stays put [47.00ms]
(fail) POST /session/:id/state > a stopped run cannot be started again [47.00ms]
(fail) POST /session/:id/state > a duplicate live report does not extend a run somebody is billed for [44.00ms]
(fail) POST /session/:id/state > a state nobody defined is a validation error, not a conflict [49.00ms]
(fail) POST /session/:id/state > a person cannot report a state on their own session [48.00ms]
(fail) POST /session/:id/ticket > a later ticket replaces the first, because it is a better address [44.00ms]
(fail) POST /session/:id/ticket > a different host cannot publish an address for someone else’s run [75.00ms]
(fail) POST /session/:id/ticket > a stopped run has no address to publish [48.00ms]
(fail) POST /session/:id/ticket > a ticket has to say something [47.00ms]
(fail) Session routes in the spec > every path a caller needs is documented [48.00ms]
error: Cannot find module './placement.js' from '/home/…/packages/core/src/box/placement.test.ts'
(fail) Session jobs > a requested session is the job, and it carries its kind [37.00ms]
(fail) Session jobs > a job belongs to the machine its box is placed on and to no other [71.00ms]
(fail) Session jobs > only a requested session is work; a claimed one is not offered again [34.00ms]
(fail) Session claim > the claim is a compare-and-set, so the second attempt finds nothing to move [37.00ms]
(fail) Session claim > a machine that is not the box’s host cannot move the row [72.00ms]
(fail) Session claim > a session that does not exist is refused the same way as one that is not yours [31.00ms]
(fail) Session claim > re-reporting the state you already reported changes nothing [37.00ms]
(fail) Session claim > a transition off the table is refused and the row does not move [36.00ms]
(fail) Session claim > the timestamps survive a duplicate report, which is what billing rests on [33.00ms]
(fail) Session claim > publishing a ticket is scoped to the host too [73.00ms]
(fail) Session claim > a stopped session has no address to publish [36.00ms]
7 pass
39 fail
1 error
Ran 46 tests across 3 files. [2.88s]
```
(46 rather than 49 because the three `Placement` tests never ran — the
module
they import did not exist.)
## And passing after
```
49 pass
0 fail
126 expect() calls
Ran 49 tests across 3 files. [3.72s]
```
Full suite, against a fresh migrated database on `DATABASE_URL` and
`TEST_DATABASE_URL`:
```
181 pass
0 fail
479 expect() calls
Ran 181 tests across 16 files. [7.89s]
```
Baseline before this branch was `138 pass, 0 fail, 371 expect() calls`
across
14 files, so 43 tests and 108 assertions are new and nothing regressed.
`tsc --noEmit -p apps/api` reports the same five pre-existing errors in
`app/utils/hook.ts` and `app/utils/validator.ts` as it does on `dev`,
and none
in the files this branch adds.
## Shared files touched
- `apps/api/app/index.ts` — three lines: one import, and two mounts
(`/session`, plus `SessionApi.machineRoute` at `/machine`).
- `packages/core/src/box/index.ts` — one import and `Box.createPlaced`
added.
Nothing existing changed.
- `packages/core/src/session/session.test.ts` — `scene()` now also
returns
`machineId`; two new `describe` blocks appended.
- `packages/core/src/examples.ts` was **not** touched; the job schema's
examples reuse the existing `Examples.Session`, `Examples.Box` and
`Examples.Game` values.
No migration was created and none was needed.
## Judgement calls
1. **`GET /machine/jobs` lives in `session.ts`, mounted at `/machine`.**
The
path belongs under the prefix a host already uses for everything it asks
about itself, but the handler is this lane's code, so it is a second
Hono
instance (`SessionApi.machineRoute`) rather than an edit to
`machine.ts`.
2. **A job's payload beyond `kind` was not specified anywhere**, so it
is:
`kind`, `sessionId`, `boxId`, `boxTier`, `gameId`, `steamAppId`,
`linkedAccountId`. `steamAppId` is in there because the agent launches
by
store id and not by our internal one, and `boxTier` because the tier
also
sets output geometry. **This is the most likely place for the two
implementations to disagree** — see the disagreement note below.
3. **A run's terminal states refuse a ticket** (`409`). Nothing said
what
publishing an address for a stopped run should do; writing one can only
mislead a client that is still polling.
4. **A state report accepts only `starting`, `live`, `ended`,
`failed`.**
`requested` is written once, at creation, so reporting it is a `400`
rather
than a no-op.
5. **A box already running refuses a second run** with `409`. Not stated
in so
many words, but `Session.activeForBox` documents itself as the query
that
should start refusing rather than picking a winner silently, and this is
the
caller that would otherwise have made it pick.
6. **`Placement.onlyHost` refuses when there is more than one
candidate**
rather than taking the first row. Picking would be a scheduling policy
invented by accident and impossible to find later. It also refuses when
there are none, because `box.machineId` is not nullable and a placer
that
cannot answer must say so.
7. **`POST /session` returns `201`**, and its body is `.strict()`.
8. **Person-facing 404s, agent-facing 403s.** Both are the settled shape
for
their realm and both are asserted identical between "does not exist" and
"not yours".
9. **`POST /session` accepts an explicit `linkedAccountId`**, defaulting
to the
one the caller's session carries. A personal access token carries none,
so
requiring the session to supply it would make that credential unable to
start a run; the account is verified to belong to the caller either way.
## Where the specification was ambiguous or came out wrong
- **The lost-claim 409 is not reachable through the HTTP endpoint as
specified.** A box names exactly one machine, so both racers for a given
run
authenticate as *the same* machine — and the same machine re-reporting a
state it already reported is specified as `200`. Which of the two
answers a
caller gets therefore depends only on whether its read happened before
or
after the winner's write: concurrent gets `409`, a sequential retry gets
`200`. That is a defensible reading and it is what is implemented, but
it
means the two rules are distinguished by timing and not by anything a
caller
can see. If a second agent process per host is ever real, this needs a
claim
token in the request rather than a timing accident.
- **The job payload is unspecified**, which is the one part of the wire
the
agent parses and the one thing a document written before two
implementations
exist was supposed to pin. If the other end's test expects different
field
names, that is this gap and not a bug in either implementation, and it
should
be settled in the specification before either side moves.
- **Nothing said whether a person may report a state.** Implemented as
`403`:
terminal states are the agent's to write, and a person closing their app
is
a different fact from a run that stopped. That reading is also why
cancellation, listed below, has
nowhere to live yet.
## What this does not verify
- **No live round trip.** Every assertion here goes through
`app.request`
against a real Postgres. No real host agent has ever called any of these
five endpoints, and no client has ever read a ticket out of one.
- **The claim is not verified under concurrency.** The compare-and-set
is
pinned by stepping two attempts in sequence and showing the second
matches
zero rows. Two agents actually racing, on two connections, is not tested
and
cannot be from a single-process test.
- **The wire shape is asserted against this lane's reading of the
specification, not against the other end.** That is deliberate — each
end
writes its own test — but it means a shape agreement has *not* been
demonstrated. Both tests passing separately is the evidence, and it does
not
exist yet.
- **No ticket in this repository has ever been a real iroh ticket.**
They are
opaque strings to every assertion here; that the value survives a round
trip
says nothing about whether anything could connect to it.
- **`steamAppId` reaching the agent is untested end to end.** It is
asserted
present and correct in the poll response and nothing consumes it.
- **Nothing reaps a stuck `requested`.** A run whose host never polls
sits
there forever, and no screen distinguishes that from "starting soon".
Left
open deliberately: a timeout here would hide the gap rather than close
it.
- **Cancellation has no path.** A person who closes their app between
`requested` and `starting` leaves work that will be claimed and started
into
an empty room. There is no endpoint for it and this branch did not
invent
one.
- **Nobody writes `ended` when the agent itself dies.** The only writer
of
terminal states is the agent, so a host that loses power leaves a `live`
run
that keeps billing. The fix belongs with whatever decides host liveness
has
authority over a run's state, and that does not exist.
`Machine.isOnline`
exists but nothing joins it to a session.
- **Placement is verified only for the single-host case and the two
refusals.**
No test exercises a real alternative placer beyond an inline stub, and
nothing calls `Box.createPlaced` from an endpoint, because there is no
box
creation endpoint yet.
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
This PR adds the HTTP and core-domain session lifecycle, including user
session requests, machine job polling, host-scoped state transitions and
ticket publication, placement support, box-state synchronization, and a
database constraint ensuring one active session per box.
- Adds authenticated user and machine session endpoints with
resource-level ownership and host scoping.
- Uses compare-and-set state transitions so only one caller can claim a
requested session.
- Adds a partial unique index and duplicate-data repair migration to
enforce one active run per box.
- Clears connection tickets when sessions become terminal.
- Adds placement abstractions and extensive API, domain, migration, and
concurrency-oriented tests.
- The latest changes document and test that ownership is checked per
user rather than per selected linked account; they do not resolve the
previously reported multi-account launch failure.
<h3>Confidence Score: 4/5</h3>
The PR is not yet safe to merge because selecting a different linked
account can still launch a game that only another account owns.
The previously reported ownership issue remains unresolved: the current
request path checks the game against the user-wide library and
separately verifies only that the selected linked account belongs to
that user. The new test explicitly accepts this behavior, so a user with
multiple Steam links can request a game through an account that does not
own it, causing the host launch to fail. The three resolved previous
findings are fully addressed or manually resolved and do not remain
outstanding.
**Files Needing Attention:** apps/api/app/routes/session.ts
<h3>Important Files Changed</h3>
| Filename | Overview |
|----------|----------|
| apps/api/app/routes/session.ts | Adds user and host session endpoints
with scoped authorization and validation, but the previously reported
selected-account ownership mismatch remains. |
| packages/core/src/session/index.ts | Adds job queries, host-scoped
compare-and-set transitions, ticket lifecycle handling, box-state
synchronization, and active-session conflict translation. |
| packages/core/src/session/session.sql.ts | Declares the partial unique
index enforcing at most one undeleted, unstopped session per box. |
| packages/core/migrations/0008_session_one_active_run_per_box.sql |
Repairs pre-existing duplicate active sessions, clears their stale
tickets, and creates the active-session unique index. |
| packages/core/src/box/placement.ts | Introduces a placement seam that
selects the sole owner host and refuses ambiguous or unavailable
placement. |
| apps/api/test/session.test.ts | Adds broad endpoint coverage and now
explicitly demonstrates the unresolved per-user rather than
per-linked-account ownership behavior. |
<h3>Sequence Diagram</h3>
```mermaid
sequenceDiagram
participant U as User client
participant A as Session API
participant D as Core domain
participant DB as PostgreSQL
participant H as Host agent
U->>A: POST /session
A->>D: Validate box, game, library, and linked account
D->>DB: Insert requested session
DB-->>D: Enforce one active session per box
H->>A: GET /machine/jobs
A->>D: List requested sessions scoped to host
D->>DB: Join session through box.machineId
H->>A: POST /session/:id/state (starting)
A->>D: Compare-and-set state for host
D->>DB: Atomic scoped transition
H->>A: POST /session/:id/ticket
A->>D: Publish ticket while addressable
U->>A: GET /session/:id
A-->>U: Current state and ticket
H->>A: POST /session/:id/state (ended/failed)
D->>DB: Set terminal state and clear ticket
```
<sub>Reviews (4): Last reviewed commit: ["fix(api): say what the library
check
act..."](7bdca1240f)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=60463182)</sub>
<details><summary><h4>Context used (3)</h4></summary>
- Knowledge Base — [API HTTP composition and
authorization](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/api-http-and-auth.md)
- Knowledge Base — [Core domain and
persistence](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/core-domain-data.md)
- Knowledge Base — [Users, identity, and game
libraries](https://app.greptile.com/nestri/-/custom-context/knowledge-base/nestrilabs/nestri/-/docs/core-identity-and-library.md)
</details>
<!-- /greptile_comment -->
This commit is contained in:
@@ -16,6 +16,7 @@ import { IndexApi } from './routes/index.js';
|
||||
import { LibraryApi } from './routes/library.js';
|
||||
import { MachineApi } from './routes/machine.js';
|
||||
import { PairingCodeApi } from './routes/pairing-code.js';
|
||||
import { SessionApi } from './routes/session.js';
|
||||
import { SteamApi } from './routes/steam.js';
|
||||
import { UserApi } from './routes/user.js';
|
||||
import { WaitlistApi } from './routes/waitlist.js';
|
||||
@@ -44,6 +45,8 @@ const routes = app
|
||||
.route('/games', GameApi.route)
|
||||
.route('/pairing-code', PairingCodeApi.route)
|
||||
.route('/machine', MachineApi.route)
|
||||
.route('/machine', SessionApi.machineRoute)
|
||||
.route('/session', SessionApi.route)
|
||||
.route('/access-token', AccessTokenApi.route)
|
||||
.route('/waitlist', WaitlistApi.route)
|
||||
.onError((error, c) => {
|
||||
|
||||
375
apps/api/app/routes/session.ts
Normal file
375
apps/api/app/routes/session.ts
Normal file
@@ -0,0 +1,375 @@
|
||||
import { Actor } from '@nestri/core/actor';
|
||||
import { Box } from '@nestri/core/box/index';
|
||||
import { ErrorCodes, VisibleError } from '@nestri/core/error';
|
||||
import { Examples } from '@nestri/core/examples';
|
||||
import { Game } from '@nestri/core/game/index';
|
||||
import { Identifier } from '@nestri/core/id';
|
||||
import { Session } from '@nestri/core/session/index';
|
||||
import { Library } from '@nestri/core/user/library';
|
||||
import { LinkedAccount } from '@nestri/core/user/linked-account';
|
||||
import { Hono } from 'hono';
|
||||
import { describeRoute } from 'hono-openapi';
|
||||
import { z } from 'zod';
|
||||
|
||||
import { ErrorResponses, machineOnly, notPublic, Result, validator } from '../utils';
|
||||
|
||||
/**
|
||||
* Requesting a run, and carrying one out.
|
||||
*
|
||||
* Two very different callers meet on one resource here. A person asks for a
|
||||
* run and then watches it; the host agent is handed the work and reports what
|
||||
* happened. The rule that keeps them apart is that an agent may only see or
|
||||
* touch a run whose box is placed on its own hardware, and it is enforced in
|
||||
* the query rather than by the agent asking for its own work — a host
|
||||
* credential is a long-lived secret on hardware in somebody's home, and what
|
||||
* one leaking can reach is decided here.
|
||||
*/
|
||||
export namespace SessionApi {
|
||||
/**
|
||||
* One answer for "no such run" and "not your run".
|
||||
*
|
||||
* Both are the same refusal on purpose: an agent that could tell the
|
||||
* difference could discover which ids exist by reporting states at them.
|
||||
*/
|
||||
function notYours(): never {
|
||||
throw new VisibleError(
|
||||
'forbidden',
|
||||
ErrorCodes.Permission.FORBIDDEN,
|
||||
'No such session, or it is not on this machine'
|
||||
);
|
||||
}
|
||||
|
||||
function conflict(message: string): never {
|
||||
throw new VisibleError('already_exists', ErrorCodes.Validation.INVALID_STATE, message);
|
||||
}
|
||||
|
||||
/** The person a run belongs to, refusing a host acting as its owner. */
|
||||
function actingPerson(): string {
|
||||
const actor = Actor.use();
|
||||
if (actor.type !== 'user' && actor.type !== 'member') {
|
||||
throw new VisibleError(
|
||||
'forbidden',
|
||||
ErrorCodes.Permission.INSUFFICIENT_PERMISSIONS,
|
||||
'Requesting or reading a session requires a user session'
|
||||
);
|
||||
}
|
||||
return actor.properties.userID;
|
||||
}
|
||||
|
||||
const StateReport = z
|
||||
.object({
|
||||
state: Session.ReportableState.meta({
|
||||
description: 'Where the run has got to',
|
||||
example: 'starting'
|
||||
}),
|
||||
errorMessage: z.string().max(1024).nullable().optional().meta({
|
||||
description: 'Why it failed. Kept only for a run that did',
|
||||
example: Examples.Session.errorMessage
|
||||
})
|
||||
})
|
||||
.strict();
|
||||
|
||||
export const route = new Hono()
|
||||
.post(
|
||||
'/',
|
||||
notPublic,
|
||||
describeRoute({
|
||||
tags: ['Session'],
|
||||
summary: 'Ask for a run of a box',
|
||||
description:
|
||||
'Creates the run in state `requested`, which is the work order the box’s host picks up. This makes no decision about where the run happens: a box already names the hardware it is placed on, so the run inherits it. Poll the run to watch it start, and re-read its ticket rather than keeping the first one.',
|
||||
responses: {
|
||||
201: {
|
||||
content: { 'application/json': { schema: Result(Session.Info) } },
|
||||
description: 'The run has been requested'
|
||||
},
|
||||
400: ErrorResponses[400],
|
||||
401: ErrorResponses[401],
|
||||
403: ErrorResponses[403],
|
||||
404: ErrorResponses[404],
|
||||
409: ErrorResponses[409]
|
||||
}
|
||||
}),
|
||||
validator(
|
||||
'json',
|
||||
z
|
||||
.object({
|
||||
boxId: z.string().min(1).meta({
|
||||
description: 'The box to run',
|
||||
example: Examples.Session.boxId
|
||||
}),
|
||||
gameId: z.string().min(1).meta({
|
||||
description: 'The game to launch',
|
||||
example: Examples.Session.gameId
|
||||
}),
|
||||
linkedAccountId: z.string().min(1).optional().meta({
|
||||
description:
|
||||
'Which linked account is playing. Defaults to the one the caller signed in with',
|
||||
example: Examples.Session.linkedAccountId
|
||||
})
|
||||
})
|
||||
// Strict, so that naming hardware is a validation error rather
|
||||
// than a field quietly ignored. There is nothing to choose:
|
||||
// asking for a run is not where a box is placed.
|
||||
.strict()
|
||||
),
|
||||
async (c) => {
|
||||
const body = c.req.valid('json');
|
||||
const userId = actingPerson();
|
||||
|
||||
const box = await Box.fromID(body.boxId);
|
||||
if (!box || box.userId !== userId) {
|
||||
// Somebody else's box and a box that was never created are the
|
||||
// same answer, so ids cannot be probed for.
|
||||
throw new VisibleError(
|
||||
'not_found',
|
||||
ErrorCodes.NotFound.RESOURCE_NOT_FOUND,
|
||||
'No such box, or it is not yours'
|
||||
);
|
||||
}
|
||||
|
||||
const game = await Game.fromID(body.gameId);
|
||||
if (!game) {
|
||||
throw new VisibleError(
|
||||
'not_found',
|
||||
ErrorCodes.NotFound.RESOURCE_NOT_FOUND,
|
||||
'No such game'
|
||||
);
|
||||
}
|
||||
|
||||
// A game nobody has synced for this person is a box that starts,
|
||||
// tries to launch and fails minutes later with nothing to point
|
||||
// at. Refusing here is the same answer sooner.
|
||||
//
|
||||
// Told apart from a game that does not exist rather than hidden:
|
||||
// the catalog is public, so there is nothing to hide, and "you do
|
||||
// not own this" is a sentence a person can act on.
|
||||
//
|
||||
// **This is a weaker check than the one that matters.** A run
|
||||
// launches as one account, but a library entry records only the
|
||||
// person, so what is verified is "somebody this person has linked
|
||||
// owns it" and not "the account playing owns it". For the one
|
||||
// linked account most people have those are the same sentence;
|
||||
// for two they are not, and the second account can ask for a game
|
||||
// only the first owns. Answering the real question needs the
|
||||
// account recorded on the library entry, which is a decision
|
||||
// about what a library *is* and not something to infer here.
|
||||
//
|
||||
// The library is also a synced copy, so this refuses a game
|
||||
// bought since the last sync. Both gaps let a launch fail late;
|
||||
// neither is a reason to start runs already known to fail.
|
||||
const owned = await Library.findByUserAndGame({ userId, gameId: game.id });
|
||||
if (!owned) {
|
||||
throw new VisibleError(
|
||||
'forbidden',
|
||||
ErrorCodes.Permission.FORBIDDEN,
|
||||
'That game is not in your library'
|
||||
);
|
||||
}
|
||||
|
||||
const actor = Actor.use();
|
||||
const linkedAccountId =
|
||||
body.linkedAccountId ||
|
||||
(actor.type === 'user' ? actor.properties.linkedAccountID : '') ||
|
||||
'';
|
||||
if (!linkedAccountId) {
|
||||
// Which account is playing is the question the "who's playing?"
|
||||
// screen asks, and some credentials carry no answer to it. Then
|
||||
// the caller has to say.
|
||||
throw new VisibleError(
|
||||
'validation',
|
||||
ErrorCodes.Validation.MISSING_REQUIRED_FIELD,
|
||||
'Say which linked account is playing',
|
||||
'linkedAccountId'
|
||||
);
|
||||
}
|
||||
const linked = await LinkedAccount.fromID(linkedAccountId);
|
||||
if (!linked || linked.userId !== userId) {
|
||||
throw new VisibleError(
|
||||
'forbidden',
|
||||
ErrorCodes.Permission.FORBIDDEN,
|
||||
'That account is not linked to you'
|
||||
);
|
||||
}
|
||||
|
||||
// A box runs one thing at a time. Refusing is the honest answer;
|
||||
// starting a second run would leave two rows that both think they
|
||||
// own the same hardware.
|
||||
//
|
||||
// This read is the message, not the guarantee — two callers can
|
||||
// both pass it. `Session.request` is refused by a unique index on
|
||||
// the same predicate, and answers with the same 409 in the same
|
||||
// words, so which one caught it is not visible from here.
|
||||
const active = await Session.activeForBox(box.id);
|
||||
if (active) {
|
||||
conflict(Session.BOX_BUSY);
|
||||
}
|
||||
|
||||
const session = await Session.request({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: box.id,
|
||||
gameId: game.id,
|
||||
linkedAccountId
|
||||
});
|
||||
return c.json({ data: session }, 201);
|
||||
}
|
||||
)
|
||||
.get(
|
||||
'/:id',
|
||||
notPublic,
|
||||
describeRoute({
|
||||
tags: ['Session'],
|
||||
summary: 'Read a run you asked for',
|
||||
description:
|
||||
'Poll this while a run starts. The ticket appears part-way through and is republished as addresses are discovered, so re-read it rather than keeping the first one — a client that treats the first ticket as final works on a local network and fails from anywhere else. Once the run reaches a terminal state the ticket is null: stop polling and stop dialling it.',
|
||||
responses: {
|
||||
200: {
|
||||
content: { 'application/json': { schema: Result(Session.Info) } },
|
||||
description: 'The run as it stands'
|
||||
},
|
||||
401: ErrorResponses[401],
|
||||
403: ErrorResponses[403],
|
||||
404: ErrorResponses[404]
|
||||
}
|
||||
}),
|
||||
validator(
|
||||
'param',
|
||||
z.object({
|
||||
id: z.string().meta({ description: 'The run to read', example: Examples.Session.id })
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const session = await Session.forOwner({
|
||||
id: c.req.valid('param').id,
|
||||
userId: actingPerson()
|
||||
});
|
||||
if (!session) {
|
||||
// Owner-scoped in the query, so somebody else's run and one that
|
||||
// never existed answer the same way.
|
||||
throw new VisibleError(
|
||||
'not_found',
|
||||
ErrorCodes.NotFound.RESOURCE_NOT_FOUND,
|
||||
'No such session, or it is not yours'
|
||||
);
|
||||
}
|
||||
return c.json({ data: session });
|
||||
}
|
||||
)
|
||||
.post(
|
||||
'/:id/state',
|
||||
machineOnly,
|
||||
describeRoute({
|
||||
tags: ['Session'],
|
||||
summary: 'Report where a run has got to',
|
||||
description:
|
||||
'For the host the run’s box is placed on, and no other. Moving a run out of `requested` is the claim, and it is a compare-and-set: exactly one caller can take a given run, and one that loses gets 409. Re-reporting a state already reported is fine and changes nothing, including the timestamps a run is billed on. A transition that does not exist is 409 and the run does not move.',
|
||||
responses: {
|
||||
200: {
|
||||
content: { 'application/json': { schema: Result(Session.Info) } },
|
||||
description: 'The run as it stands after the report'
|
||||
},
|
||||
400: ErrorResponses[400],
|
||||
403: ErrorResponses[403],
|
||||
409: ErrorResponses[409]
|
||||
}
|
||||
}),
|
||||
validator('param', z.object({ id: z.string() })),
|
||||
validator('json', StateReport),
|
||||
async (c) => {
|
||||
const body = c.req.valid('json');
|
||||
const result = await Session.transition({
|
||||
id: c.req.valid('param').id,
|
||||
machineId: Actor.machineID,
|
||||
state: body.state,
|
||||
errorMessage: body.errorMessage ?? null
|
||||
});
|
||||
|
||||
switch (result.outcome) {
|
||||
case 'forbidden':
|
||||
notYours();
|
||||
case 'illegal':
|
||||
conflict(`A run in state ${result.session?.state} cannot become ${body.state}`);
|
||||
case 'lost':
|
||||
conflict('Another caller moved this run first');
|
||||
default:
|
||||
// `moved` and `unchanged` are both success. An agent retrying
|
||||
// after a lost response must not be told it broke something.
|
||||
return c.json({ data: result.session });
|
||||
}
|
||||
}
|
||||
)
|
||||
.post(
|
||||
'/:id/ticket',
|
||||
machineOnly,
|
||||
describeRoute({
|
||||
tags: ['Session'],
|
||||
summary: 'Publish the address a client should connect to',
|
||||
description:
|
||||
'For the host the run’s box is placed on, and no other. Republish freely: a later ticket is a better address for the same run, not a second run, and the address changes as more of them are discovered. Only a run being brought up has an address: claim it by reporting `starting` first, and expect 409 both before that and once it has stopped.',
|
||||
responses: {
|
||||
200: {
|
||||
content: { 'application/json': { schema: Result(Session.Info) } },
|
||||
description: 'The ticket is published'
|
||||
},
|
||||
400: ErrorResponses[400],
|
||||
403: ErrorResponses[403],
|
||||
409: ErrorResponses[409]
|
||||
}
|
||||
}),
|
||||
validator('param', z.object({ id: z.string() })),
|
||||
validator(
|
||||
'json',
|
||||
z
|
||||
.object({
|
||||
ticket: z.string().min(1).meta({
|
||||
description: 'The current connect ticket',
|
||||
example: Examples.Session.ticket
|
||||
})
|
||||
})
|
||||
.strict()
|
||||
),
|
||||
async (c) => {
|
||||
const result = await Session.publishTicket({
|
||||
id: c.req.valid('param').id,
|
||||
machineId: Actor.machineID,
|
||||
ticket: c.req.valid('json').ticket
|
||||
});
|
||||
|
||||
switch (result.outcome) {
|
||||
case 'forbidden':
|
||||
notYours();
|
||||
case 'unclaimed':
|
||||
conflict('Claim this run by reporting `starting` before publishing an address');
|
||||
case 'closed':
|
||||
conflict('That run has stopped, so it has no address to publish');
|
||||
default:
|
||||
return c.json({ data: result.session });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
/**
|
||||
* The host agent's side of the same resource, mounted where a host looks
|
||||
* for it: everything a box asks about itself lives under one prefix.
|
||||
*/
|
||||
export const machineRoute = new Hono().get(
|
||||
'/jobs',
|
||||
machineOnly,
|
||||
describeRoute({
|
||||
tags: ['Session'],
|
||||
summary: 'Ask for work',
|
||||
description:
|
||||
'Returns the runs waiting to be started on the calling host, and only those — the host comes from its own credentials and the scope is the query, so a box cannot see work for another. Poll at the cadence the heartbeat hands down. Each job carries its kind, so a second kind of work is an addition rather than a change of shape.',
|
||||
responses: {
|
||||
200: {
|
||||
content: { 'application/json': { schema: Result(z.array(Session.Job)) } },
|
||||
description: 'Work waiting for this host, oldest first'
|
||||
},
|
||||
403: ErrorResponses[403]
|
||||
}
|
||||
}),
|
||||
async (c) => {
|
||||
return c.json({ data: await Session.listJobsForMachine(Actor.machineID) });
|
||||
}
|
||||
);
|
||||
}
|
||||
739
apps/api/test/session.test.ts
Normal file
739
apps/api/test/session.test.ts
Normal file
@@ -0,0 +1,739 @@
|
||||
import { afterAll, describe, expect, test } from 'bun:test';
|
||||
|
||||
import { AccessToken } from '@nestri/core/access-token/index';
|
||||
import { Box } from '@nestri/core/box/index';
|
||||
import { Fixtures } from '@nestri/core/db/fixtures';
|
||||
import { testDb } from '@nestri/core/db/test';
|
||||
import { Game } from '@nestri/core/game/index';
|
||||
import { Identifier } from '@nestri/core/id';
|
||||
import { Machine } from '@nestri/core/machine/index';
|
||||
import { Session } from '@nestri/core/session/index';
|
||||
import { Library } from '@nestri/core/user/library';
|
||||
import { LinkedAccount } from '@nestri/core/user/linked-account';
|
||||
|
||||
import { app } from '../app/index';
|
||||
import './setup';
|
||||
|
||||
const sql = testDb();
|
||||
|
||||
const createdUserIds: string[] = [];
|
||||
const createdGameIds: string[] = [];
|
||||
|
||||
async function newGame(steamAppId: number): Promise<string> {
|
||||
const [row] = await Game.upsert({
|
||||
id: Identifier.ascending('game'),
|
||||
steamAppId,
|
||||
slug: `session-route-${steamAppId}`,
|
||||
name: `Session Route ${steamAppId}`
|
||||
});
|
||||
if (!row) throw new Error('expected a game row');
|
||||
createdGameIds.push(row.id);
|
||||
return row.id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything one session needs, plus both sets of credentials that reach it.
|
||||
*
|
||||
* The person authenticates with a personal token, which is the one user
|
||||
* credential a test can mint without an auth service; the host authenticates
|
||||
* as itself with the secret registration hands back exactly once.
|
||||
*/
|
||||
async function scene(label: string, steamAppId: number) {
|
||||
const owner = await Fixtures.owner(label);
|
||||
createdUserIds.push(owner.userId);
|
||||
|
||||
const registered = await Machine.register({
|
||||
id: Identifier.ascending('machine'),
|
||||
ownerUserId: owner.userId,
|
||||
teamId: owner.teamId,
|
||||
label
|
||||
});
|
||||
|
||||
const box = await Box.create({
|
||||
id: Identifier.ascending('box'),
|
||||
userId: owner.userId,
|
||||
machineId: registered.id,
|
||||
label,
|
||||
tier: 'sm'
|
||||
});
|
||||
|
||||
const pat = await AccessToken.create({
|
||||
id: Identifier.ascending('accessToken'),
|
||||
ownerUserId: owner.userId,
|
||||
// Null on purpose: a token scoped to the user alone makes the caller a
|
||||
// plain user actor, which is the credential a person browsing has.
|
||||
teamId: null,
|
||||
name: label
|
||||
});
|
||||
|
||||
const gameId = await newGame(steamAppId);
|
||||
// A run launches as a Steam account that owns the game, so the endpoint
|
||||
// refuses one outside the caller's library. Every scene here is about
|
||||
// something else, so the game is stocked.
|
||||
await Library.upsert({
|
||||
id: Identifier.ascending('userLibrary'),
|
||||
userId: owner.userId,
|
||||
gameId,
|
||||
playtime2w: null,
|
||||
playtimeForever: null,
|
||||
lastPlayed: null
|
||||
});
|
||||
|
||||
return {
|
||||
owner,
|
||||
box,
|
||||
machineId: registered.id,
|
||||
gameId,
|
||||
user: {
|
||||
authorization: `Bearer ${pat.token}`,
|
||||
'content-type': 'application/json'
|
||||
} as Record<string, string>,
|
||||
host: {
|
||||
'x-nestri-machine-id': registered.id,
|
||||
'x-nestri-machine-secret': registered.secret,
|
||||
'content-type': 'application/json'
|
||||
} as Record<string, string>
|
||||
};
|
||||
}
|
||||
|
||||
async function requestSession(s: Awaited<ReturnType<typeof scene>>) {
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: s.user,
|
||||
body: JSON.stringify({
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
const body = (await res.json()) as any;
|
||||
return { res, body };
|
||||
}
|
||||
|
||||
afterAll(async () => {
|
||||
if (createdUserIds.length > 0) {
|
||||
await sql`delete from "box" where user_id in ${sql(createdUserIds)}`;
|
||||
await sql`delete from "user" where id in ${sql(createdUserIds)}`;
|
||||
createdUserIds.length = 0;
|
||||
}
|
||||
if (createdGameIds.length > 0) {
|
||||
await sql`delete from "game" where id in ${sql(createdGameIds)}`;
|
||||
createdGameIds.length = 0;
|
||||
}
|
||||
});
|
||||
|
||||
describe('POST /session', () => {
|
||||
test('a request creates the job, in the envelope both ends read', async () => {
|
||||
const s = await scene('route-create', 5500);
|
||||
const { res, body } = await requestSession(s);
|
||||
|
||||
expect(res.status).toBe(201);
|
||||
// The field names are the contract. A rename on either side produces a
|
||||
// host that starts, reads nothing, and reports success — so the shape
|
||||
// is asserted whole rather than field by field.
|
||||
expect(Object.keys(body)).toEqual(['data']);
|
||||
expect(body.data).toEqual({
|
||||
id: body.data.id,
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: s.owner.linkedAccountId,
|
||||
state: 'requested',
|
||||
ticket: null,
|
||||
timeStarted: null,
|
||||
timeStopped: null,
|
||||
errorMessage: null
|
||||
});
|
||||
expect(body.data.id.startsWith('ses_')).toBe(true);
|
||||
});
|
||||
|
||||
test('creating a session makes no placement decision', async () => {
|
||||
const s = await scene('route-noplacement', 5501);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
// A session inherits its machine through its box, so there is nothing
|
||||
// to choose here and no way for a caller to ask for a host.
|
||||
expect(body.data).not.toHaveProperty('machineId');
|
||||
|
||||
const withHost = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: s.user,
|
||||
body: JSON.stringify({
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: s.owner.linkedAccountId,
|
||||
machineId: s.machineId
|
||||
})
|
||||
});
|
||||
expect(withHost.status).toBe(400);
|
||||
});
|
||||
|
||||
test('a box somebody else owns is not there to run', async () => {
|
||||
const mine = await scene('route-mine', 5502);
|
||||
const theirs = await scene('route-theirs', 5503);
|
||||
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: mine.user,
|
||||
body: JSON.stringify({
|
||||
boxId: theirs.box.id,
|
||||
gameId: mine.gameId,
|
||||
linkedAccountId: mine.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
expect(res.status).toBe(404);
|
||||
|
||||
const unknown = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: mine.user,
|
||||
body: JSON.stringify({
|
||||
boxId: Identifier.ascending('box'),
|
||||
gameId: mine.gameId,
|
||||
linkedAccountId: mine.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
// Owner-scoped in the query, so somebody else's box and a box that was
|
||||
// never created are the same answer.
|
||||
expect(unknown.status).toBe(404);
|
||||
expect(await res.json()).toEqual(await unknown.json());
|
||||
});
|
||||
|
||||
test('a box already running refuses a second run rather than picking one', async () => {
|
||||
const s = await scene('route-busy', 5504);
|
||||
expect((await requestSession(s)).res.status).toBe(201);
|
||||
|
||||
const second = await requestSession(s);
|
||||
expect(second.res.status).toBe(409);
|
||||
expect(second.body.type).toBe('already_exists');
|
||||
});
|
||||
|
||||
test('two requests racing for one box still start it once', async () => {
|
||||
const s = await scene('route-race', 5509);
|
||||
const [a, b] = await Promise.all([requestSession(s), requestSession(s)]);
|
||||
|
||||
// Which request wins is a timing detail; that exactly one does is not.
|
||||
// The pre-check and the unique index answer identically, so the loser
|
||||
// cannot tell which caught it.
|
||||
//
|
||||
// This asserts the endpoint's answer, not the invariant: two requests
|
||||
// in one process usually interleave such that the pre-check catches
|
||||
// the second, so it passes with the unique index dropped. The index is
|
||||
// pinned in the core tests, where both callers can be made to read
|
||||
// before either writes.
|
||||
const statuses = [a.res.status, b.res.status].sort();
|
||||
expect(statuses).toEqual([201, 409]);
|
||||
expect([a.body, b.body].find((x) => x.type)?.type).toBe('already_exists');
|
||||
|
||||
expect(await Session.listByBox(s.box.id)).toHaveLength(1);
|
||||
// The failure this prevents: the host offered the same box twice.
|
||||
const jobs = await app.request('/machine/jobs', { headers: s.host });
|
||||
expect(((await jobs.json()) as any).data).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('you can only play as an account you have linked', async () => {
|
||||
const mine = await scene('route-account-mine', 5505);
|
||||
const theirs = await scene('route-account-theirs', 5506);
|
||||
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: mine.user,
|
||||
body: JSON.stringify({
|
||||
boxId: mine.box.id,
|
||||
gameId: mine.gameId,
|
||||
linkedAccountId: theirs.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
|
||||
test('you can only run a game you own', async () => {
|
||||
const s = await scene('route-unowned', 5560);
|
||||
// A real game in the catalog, simply not in this person's library.
|
||||
const unowned = await newGame(5561);
|
||||
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: s.user,
|
||||
body: JSON.stringify({
|
||||
boxId: s.box.id,
|
||||
gameId: unowned,
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
// Told apart from a game that does not exist, deliberately: the catalog
|
||||
// is public, so there is nothing to hide, and a box that starts and
|
||||
// then cannot launch is a worse answer minutes later.
|
||||
expect(res.status).toBe(403);
|
||||
expect(await Session.listByBox(s.box.id)).toHaveLength(0);
|
||||
});
|
||||
|
||||
test('the library check is per person, not per account it plays as', async () => {
|
||||
const s = await scene('route-multilink', 5562);
|
||||
// A second Steam account on the same person. The unique index is on
|
||||
// (provider, providerAccountId) and is global rather than per user, so
|
||||
// nothing stops this — but a fixed id here would collide with its own
|
||||
// previous run, hence one shaped like a SteamID64 and unique per run.
|
||||
const second = await LinkedAccount.create({
|
||||
id: Identifier.ascending('linkedAccount'),
|
||||
userId: s.owner.userId,
|
||||
provider: 'steam',
|
||||
providerAccountId: `7656119${Date.now()}`.slice(0, 17),
|
||||
profile: null
|
||||
});
|
||||
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: s.user,
|
||||
body: JSON.stringify({
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: second
|
||||
})
|
||||
});
|
||||
|
||||
// Accepted, and this pins a known gap rather than asserting it is
|
||||
// right: a library entry records the person and not the account the
|
||||
// games came from, so "the account playing owns this" cannot be asked.
|
||||
// The run will fail at launch exactly as it did before the check
|
||||
// existed. Closing it means recording the account on the library
|
||||
// entry, which changes what a library is and what the sync must send.
|
||||
expect(res.status).toBe(201);
|
||||
});
|
||||
|
||||
test('an unknown game is a 404 and not a foreign key crash', async () => {
|
||||
const s = await scene('route-nogame', 5507);
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: s.user,
|
||||
body: JSON.stringify({
|
||||
boxId: s.box.id,
|
||||
gameId: Identifier.ascending('game'),
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
expect(res.status).toBe(404);
|
||||
});
|
||||
|
||||
test('a host cannot ask for a session on its owner’s behalf', async () => {
|
||||
const s = await scene('route-hostcreate', 5508);
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
})
|
||||
});
|
||||
// A box holds credentials but is not the person who owns it.
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
|
||||
test('requesting a session requires a signed-in person', async () => {
|
||||
const res = await app.request('/session', {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({ boxId: 'box_x', gameId: 'gam_x', linkedAccountId: 'lac_x' })
|
||||
});
|
||||
expect(res.status).toBe(401);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GET /session/:id', () => {
|
||||
test('the owner reads their own run, ticket and all', async () => {
|
||||
const s = await scene('route-read', 5510);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket: 'nodeaaa-one' })
|
||||
});
|
||||
|
||||
const res = await app.request(`/session/${body.data.id}`, { headers: s.user });
|
||||
expect(res.status).toBe(200);
|
||||
const read = (await res.json()) as any;
|
||||
expect(read.data.state).toBe('starting');
|
||||
// A ticket may appear while the state is still `starting`, and the
|
||||
// client is expected to re-read rather than cache the first one.
|
||||
expect(read.data.ticket).toBe('nodeaaa-one');
|
||||
});
|
||||
|
||||
test('somebody else’s run is not visible, and neither is its absence', async () => {
|
||||
const mine = await scene('route-read-mine', 5511);
|
||||
const theirs = await scene('route-read-theirs', 5512);
|
||||
const { body } = await requestSession(theirs);
|
||||
|
||||
const forbidden = await app.request(`/session/${body.data.id}`, { headers: mine.user });
|
||||
const unknown = await app.request(`/session/${Identifier.ascending('session')}`, {
|
||||
headers: mine.user
|
||||
});
|
||||
expect(forbidden.status).toBe(404);
|
||||
expect(unknown.status).toBe(404);
|
||||
expect(await forbidden.json()).toEqual(await unknown.json());
|
||||
});
|
||||
|
||||
test('reading a run requires a signed-in person', async () => {
|
||||
const res = await app.request('/session/ses_whatever');
|
||||
expect(res.status).toBe(401);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GET /machine/jobs', () => {
|
||||
test('a host is handed the work for its own boxes, with the kind on the wire', async () => {
|
||||
const s = await scene('route-jobs', 5520);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
const res = await app.request('/machine/jobs', { headers: s.host });
|
||||
expect(res.status).toBe(200);
|
||||
const jobs = (await res.json()) as any;
|
||||
expect(Object.keys(jobs)).toEqual(['data']);
|
||||
expect(jobs.data).toHaveLength(1);
|
||||
expect(jobs.data[0]).toEqual({
|
||||
kind: 'session.start',
|
||||
sessionId: body.data.id,
|
||||
boxId: s.box.id,
|
||||
boxTier: 'sm',
|
||||
gameId: s.gameId,
|
||||
steamAppId: 5520,
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
});
|
||||
});
|
||||
|
||||
test('a host never sees work for a box on other hardware', async () => {
|
||||
const mine = await scene('route-jobs-mine', 5521);
|
||||
const theirs = await scene('route-jobs-theirs', 5522);
|
||||
await requestSession(theirs);
|
||||
|
||||
const res = await app.request('/machine/jobs', { headers: mine.host });
|
||||
expect(res.status).toBe(200);
|
||||
// Scoped in the query rather than by the host asking for its own work.
|
||||
expect(((await res.json()) as any).data).toEqual([]);
|
||||
});
|
||||
|
||||
test('bad credentials are indistinguishable from none', async () => {
|
||||
const s = await scene('route-jobs-auth', 5523);
|
||||
const wrong = await app.request('/machine/jobs', {
|
||||
headers: { ...s.host, 'x-nestri-machine-secret': 'msk_wrong' }
|
||||
});
|
||||
const none = await app.request('/machine/jobs');
|
||||
expect(wrong.status).toBe(403);
|
||||
expect(none.status).toBe(403);
|
||||
// Bad credentials fall through to public and are then forbidden, so
|
||||
// probing tells an attacker nothing. Asserting the two are identical is
|
||||
// the only way that stays true.
|
||||
expect(await wrong.json()).toEqual(await none.json());
|
||||
});
|
||||
|
||||
test('a person cannot poll for jobs', async () => {
|
||||
const s = await scene('route-jobs-person', 5524);
|
||||
const res = await app.request('/machine/jobs', { headers: s.user });
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
});
|
||||
|
||||
describe('POST /session/:id/state', () => {
|
||||
test('the claim moves the row, and the job stops being offered', async () => {
|
||||
const s = await scene('route-claim', 5530);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
const res = await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
expect(((await res.json()) as any).data.state).toBe('starting');
|
||||
|
||||
const jobs = await app.request('/machine/jobs', { headers: s.host });
|
||||
expect(((await jobs.json()) as any).data).toEqual([]);
|
||||
});
|
||||
|
||||
test('the same host re-reporting a state it already reported is fine', async () => {
|
||||
const s = await scene('route-claim-retry', 5531);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
const report = () =>
|
||||
app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
|
||||
expect((await report()).status).toBe(200);
|
||||
// An agent retrying after a lost response must not be told it broke
|
||||
// something.
|
||||
const again = await report();
|
||||
expect(again.status).toBe(200);
|
||||
expect(((await again.json()) as any).data.state).toBe('starting');
|
||||
});
|
||||
|
||||
test('a different host reporting anything is refused, and learns nothing', async () => {
|
||||
const mine = await scene('route-claim-mine', 5532);
|
||||
const theirs = await scene('route-claim-theirs', 5533);
|
||||
const { body } = await requestSession(theirs);
|
||||
|
||||
const other = await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: mine.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
const unknown = await app.request(`/session/${Identifier.ascending('session')}/state`, {
|
||||
method: 'POST',
|
||||
headers: mine.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
|
||||
expect(other.status).toBe(403);
|
||||
expect(unknown.status).toBe(403);
|
||||
expect(await other.json()).toEqual(await unknown.json());
|
||||
expect((await Session.fromID(body.data.id))?.state).toBe('requested');
|
||||
});
|
||||
|
||||
test('a transition that is not allowed is a conflict, and the row stays put', async () => {
|
||||
const s = await scene('route-claim-illegal', 5534);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
const skipped = await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'live' })
|
||||
});
|
||||
expect(skipped.status).toBe(409);
|
||||
expect((await Session.fromID(body.data.id))?.state).toBe('requested');
|
||||
});
|
||||
|
||||
test('a stopped run cannot be started again', async () => {
|
||||
const s = await scene('route-claim-terminal', 5535);
|
||||
const { body } = await requestSession(s);
|
||||
const report = (state: string, errorMessage?: string) =>
|
||||
app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state, errorMessage })
|
||||
});
|
||||
|
||||
expect((await report('starting')).status).toBe(200);
|
||||
expect((await report('failed', 'the guest never came up')).status).toBe(200);
|
||||
expect((await report('starting')).status).toBe(409);
|
||||
|
||||
const failed = await Session.fromID(body.data.id);
|
||||
expect(failed?.state).toBe('failed');
|
||||
expect(failed?.errorMessage).toBe('the guest never came up');
|
||||
});
|
||||
|
||||
test('a duplicate live report does not extend a run somebody is billed for', async () => {
|
||||
const s = await scene('route-claim-billing', 5536);
|
||||
const { body } = await requestSession(s);
|
||||
const report = (state: string) =>
|
||||
app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state })
|
||||
});
|
||||
|
||||
await report('starting');
|
||||
const live = (await (await report('live')).json()) as any;
|
||||
expect(live.data.timeStarted).not.toBeNull();
|
||||
|
||||
const again = (await (await report('live')).json()) as any;
|
||||
expect(again.data.timeStarted).toBe(live.data.timeStarted);
|
||||
});
|
||||
|
||||
test('a state nobody defined is a validation error, not a conflict', async () => {
|
||||
const s = await scene('route-claim-bogus', 5537);
|
||||
const { body } = await requestSession(s);
|
||||
const res = await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'exploded' })
|
||||
});
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
|
||||
test('a person cannot report a state on their own session', async () => {
|
||||
const s = await scene('route-claim-person', 5538);
|
||||
const { body } = await requestSession(s);
|
||||
const res = await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.user,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
// Terminal states are written by the agent alone; a person closing the
|
||||
// app is not the same fact as a run that stopped.
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
});
|
||||
|
||||
describe('POST /session/:id/ticket', () => {
|
||||
test('a later ticket replaces the first, because it is a better address', async () => {
|
||||
const s = await scene('route-ticket', 5540);
|
||||
const { body } = await requestSession(s);
|
||||
await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
|
||||
const publish = (ticket: string) =>
|
||||
app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket })
|
||||
});
|
||||
|
||||
const first = await publish('nodeaaa-one');
|
||||
expect(first.status).toBe(200);
|
||||
expect(((await first.json()) as any).data.ticket).toBe('nodeaaa-one');
|
||||
|
||||
const second = await publish('nodeaaa-two');
|
||||
expect(((await second.json()) as any).data.ticket).toBe('nodeaaa-two');
|
||||
expect(await Session.listByBox(s.box.id)).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('a run nobody has claimed has no address to publish', async () => {
|
||||
const s = await scene('route-ticket-early', 5546);
|
||||
const { body } = await requestSession(s);
|
||||
|
||||
const early = await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket: 'nodeaaa-too-soon' })
|
||||
});
|
||||
// Publishing before reporting `starting` means the agent skipped the
|
||||
// claim, which is the only mutual exclusion in the design.
|
||||
expect(early.status).toBe(409);
|
||||
expect((await Session.fromID(body.data.id))?.ticket).toBeNull();
|
||||
|
||||
await app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state: 'starting' })
|
||||
});
|
||||
const now = await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket: 'nodeaaa-in-time' })
|
||||
});
|
||||
expect(now.status).toBe(200);
|
||||
});
|
||||
|
||||
test('a different host cannot publish an address for someone else’s run', async () => {
|
||||
const mine = await scene('route-ticket-mine', 5541);
|
||||
const theirs = await scene('route-ticket-theirs', 5542);
|
||||
const { body } = await requestSession(theirs);
|
||||
|
||||
const res = await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: mine.host,
|
||||
body: JSON.stringify({ ticket: 'nodeaaa-stolen' })
|
||||
});
|
||||
expect(res.status).toBe(403);
|
||||
expect((await Session.fromID(body.data.id))?.ticket).toBeNull();
|
||||
});
|
||||
|
||||
test('a stopped run has no address to publish', async () => {
|
||||
const s = await scene('route-ticket-dead', 5543);
|
||||
const { body } = await requestSession(s);
|
||||
const report = (state: string) =>
|
||||
app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state })
|
||||
});
|
||||
await report('starting');
|
||||
await report('live');
|
||||
await report('ended');
|
||||
|
||||
const res = await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket: 'nodeaaa-late' })
|
||||
});
|
||||
expect(res.status).toBe(409);
|
||||
expect((await Session.fromID(body.data.id))?.ticket).toBeNull();
|
||||
});
|
||||
|
||||
test('a run that stops loses the address it published', async () => {
|
||||
const s = await scene('route-ticket-cleared', 5545);
|
||||
const { body } = await requestSession(s);
|
||||
const report = (state: string) =>
|
||||
app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state })
|
||||
});
|
||||
await report('starting');
|
||||
await report('live');
|
||||
const published = await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket: 'nodeaaa-live' })
|
||||
});
|
||||
expect(((await published.json()) as any).data.ticket).toBe('nodeaaa-live');
|
||||
|
||||
await report('ended');
|
||||
|
||||
// The polling client is the reason. It reads this endpoint until it has
|
||||
// an address, and an address left behind by a run that stopped is one
|
||||
// it would dial — while publishing a replacement is already refused.
|
||||
const read = await app.request(`/session/${body.data.id}`, { headers: s.user });
|
||||
const after = (await read.json()) as any;
|
||||
expect(after.data.state).toBe('ended');
|
||||
expect(after.data.ticket).toBeNull();
|
||||
});
|
||||
|
||||
test('a ticket has to say something', async () => {
|
||||
const s = await scene('route-ticket-empty', 5544);
|
||||
const { body } = await requestSession(s);
|
||||
const res = await app.request(`/session/${body.data.id}/ticket`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ ticket: '' })
|
||||
});
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
});
|
||||
|
||||
describe('The box a run happens on', () => {
|
||||
test('the endpoints move the box, not just the run', async () => {
|
||||
const s = await scene('route-box-state', 5550);
|
||||
const { body } = await requestSession(s);
|
||||
const report = (state: string, errorMessage?: string) =>
|
||||
app.request(`/session/${body.data.id}/state`, {
|
||||
method: 'POST',
|
||||
headers: s.host,
|
||||
body: JSON.stringify({ state, errorMessage })
|
||||
});
|
||||
|
||||
expect((await Box.fromID(s.box.id))?.state).toBe('created');
|
||||
|
||||
await report('starting');
|
||||
await report('live');
|
||||
// The screens that tell a person what their hardware is doing read the
|
||||
// box, so a live run has to be visible there and not only on the run.
|
||||
expect((await Box.fromID(s.box.id))?.state).toBe('running');
|
||||
|
||||
await report('failed', 'the guest never came up');
|
||||
const stopped = await Box.fromID(s.box.id);
|
||||
expect(stopped?.state).toBe('stopped');
|
||||
expect(stopped?.stopClean).toBe(false);
|
||||
expect(stopped?.stopReason).toBe('the guest never came up');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session routes in the spec', () => {
|
||||
test('every path a caller needs is documented', async () => {
|
||||
const res = await app.request('/doc');
|
||||
const paths = Object.keys(((await res.json()) as any).paths);
|
||||
expect(paths).toContain('/session');
|
||||
expect(paths).toContain('/session/{id}');
|
||||
expect(paths).toContain('/session/{id}/state');
|
||||
expect(paths).toContain('/session/{id}/ticket');
|
||||
expect(paths).toContain('/machine/jobs');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,36 @@
|
||||
-- A box runs one thing at a time, and now the database is what says so.
|
||||
--
|
||||
-- `POST /session` reads `session.activeForBox` and refuses when something is
|
||||
-- already running, but the read and the insert are two statements. Two requests
|
||||
-- that both read "nothing is running" before either inserts each get a row, and
|
||||
-- `/machine/jobs` then hands the host the same box to start twice. A partial
|
||||
-- unique index on the same predicate the read asks about makes the second
|
||||
-- insert fail instead. ref(d-0048)
|
||||
--
|
||||
-- The index cannot be created while any box already has two unstopped runs, so
|
||||
-- the duplicates are resolved first. Keeping the newest is the only choice that
|
||||
-- matches what a person saw: their most recent request is the one they are
|
||||
-- waiting on. The older rows are stopped rather than deleted, because a session
|
||||
-- is the billing unit and rows that were once real do not vanish from it.
|
||||
--
|
||||
-- `ended` and not `failed`: nothing about these runs failed. They were work
|
||||
-- nobody picked up, and `failed` carries a reason there is none of.
|
||||
--
|
||||
-- The ticket goes with the state, exactly as the terminal transitions in
|
||||
-- `Session` do it. A duplicate that reached `starting` or `live` may have
|
||||
-- published an address, and a stopped run that still answers with one is an
|
||||
-- address a polling client would dial — with publishing a replacement already
|
||||
-- refused, it would also be the last word.
|
||||
UPDATE "session" s
|
||||
SET "state" = 'ended', "time_stopped" = now(), "ticket" = NULL
|
||||
WHERE s."time_stopped" IS NULL
|
||||
AND s."time_deleted" IS NULL
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM "session" newer
|
||||
WHERE newer."box_id" = s."box_id"
|
||||
AND newer."time_stopped" IS NULL
|
||||
AND newer."time_deleted" IS NULL
|
||||
AND (newer."time_created", newer."id") > (s."time_created", s."id")
|
||||
);--> statement-breakpoint
|
||||
|
||||
CREATE UNIQUE INDEX "session_box_active_unique" ON "session" USING btree ("box_id") WHERE time_stopped is null and time_deleted is null;
|
||||
2299
packages/core/migrations/meta/0008_snapshot.json
Normal file
2299
packages/core/migrations/meta/0008_snapshot.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -57,6 +57,13 @@
|
||||
"when": 1788460224524,
|
||||
"tag": "0007_box_session_team_notnull",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 8,
|
||||
"version": "7",
|
||||
"when": 1788547836146,
|
||||
"tag": "0008_session_one_active_run_per_box",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import { Database } from '../db/index.js';
|
||||
import { Examples } from '../examples.js';
|
||||
import { fn } from '../fn.js';
|
||||
import { BoxState, BoxTable, BoxTier } from './box.sql.js';
|
||||
import { Placement } from './placement.js';
|
||||
|
||||
/**
|
||||
* A VM someone owns.
|
||||
@@ -77,6 +78,29 @@ export namespace Box {
|
||||
}
|
||||
);
|
||||
|
||||
/**
|
||||
* Create a box and let something else decide where it runs.
|
||||
*
|
||||
* The placement seam is here, at creation, and nowhere else: `machineId` is
|
||||
* set once and every later question about which hardware a box — or a run
|
||||
* of it — belongs to is answered by joining through this row. A caller that
|
||||
* knows the host still uses `create`; a caller acting for a person does not
|
||||
* know and must not guess, which is what this overload is for.
|
||||
*/
|
||||
export const createPlaced = async (
|
||||
input: { id: string; userId: string; label: string; tier: Info['tier'] },
|
||||
placer?: Placement.Placer
|
||||
) => {
|
||||
const machineId = await Placement.choose({ userId: input.userId, tier: input.tier }, placer);
|
||||
return create({
|
||||
id: input.id,
|
||||
userId: input.userId,
|
||||
machineId,
|
||||
label: input.label,
|
||||
tier: input.tier
|
||||
});
|
||||
};
|
||||
|
||||
export const fromID = fn(Info.shape.id, async (id) => {
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
|
||||
90
packages/core/src/box/placement.test.ts
Normal file
90
packages/core/src/box/placement.test.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
import { afterAll, describe, expect, test } from 'bun:test';
|
||||
|
||||
import { Fixtures } from '../db/fixtures.js';
|
||||
import { testDb } from '../db/test.js';
|
||||
import { Identifier } from '../id.js';
|
||||
import { Placement } from './placement.js';
|
||||
import { Box } from './index.js';
|
||||
|
||||
const sql = testDb();
|
||||
|
||||
const createdUserIds: string[] = [];
|
||||
|
||||
async function newOwner(label: string) {
|
||||
const o = await Fixtures.owner(label);
|
||||
createdUserIds.push(o.userId);
|
||||
return o;
|
||||
}
|
||||
|
||||
afterAll(async () => {
|
||||
if (createdUserIds.length > 0) {
|
||||
await sql`delete from "box" where user_id in ${sql(createdUserIds)}`;
|
||||
await sql`delete from "user" where id in ${sql(createdUserIds)}`;
|
||||
createdUserIds.length = 0;
|
||||
}
|
||||
});
|
||||
|
||||
describe('Placement', () => {
|
||||
test('a box is placed when it is created, and the caller names no host', async () => {
|
||||
const owner = await newOwner('place-one');
|
||||
const machineId = await Fixtures.machine(owner, 'place-one-host');
|
||||
|
||||
// No `machineId` in the input: choosing the host is the placer's job,
|
||||
// and the whole point of the interface is that the caller cannot do it.
|
||||
const box = await Box.createPlaced({
|
||||
id: Identifier.ascending('box'),
|
||||
userId: owner.userId,
|
||||
label: 'living room',
|
||||
tier: 'sm'
|
||||
});
|
||||
|
||||
expect(box.machineId).toBe(machineId);
|
||||
});
|
||||
|
||||
test('nowhere to put it is an answer, not a crash', async () => {
|
||||
const owner = await newOwner('place-none');
|
||||
|
||||
// `box.machineId` is notNull, so a placer with no candidate must refuse
|
||||
// rather than hand back something the insert would reject.
|
||||
await expect(
|
||||
Placement.choose({ userId: owner.userId, tier: 'sm' })
|
||||
).rejects.toThrow();
|
||||
});
|
||||
|
||||
test('more than one candidate is refused rather than picked silently', async () => {
|
||||
const owner = await newOwner('place-two');
|
||||
await Fixtures.machine(owner, 'place-two-a');
|
||||
await Fixtures.machine(owner, 'place-two-b');
|
||||
|
||||
// There is no policy for choosing between hosts yet. Inventing one here
|
||||
// is how a placement decision ends up buried in the caller: the refusal
|
||||
// is what keeps the choice in one replaceable place.
|
||||
await expect(
|
||||
Placement.choose({ userId: owner.userId, tier: 'sm' })
|
||||
).rejects.toThrow();
|
||||
});
|
||||
|
||||
test('the placer is swappable without touching box creation', async () => {
|
||||
const owner = await newOwner('place-swap');
|
||||
const machineId = await Fixtures.machine(owner, 'place-swap-host');
|
||||
|
||||
const asked: unknown[] = [];
|
||||
const box = await Box.createPlaced(
|
||||
{
|
||||
id: Identifier.ascending('box'),
|
||||
userId: owner.userId,
|
||||
label: 'bedroom',
|
||||
tier: 'lg'
|
||||
},
|
||||
async (input) => {
|
||||
asked.push(input);
|
||||
return machineId;
|
||||
}
|
||||
);
|
||||
|
||||
expect(box.machineId).toBe(machineId);
|
||||
// The placer is told who the box is for and what size was asked for,
|
||||
// which is the whole input a real scheduler needs.
|
||||
expect(asked).toEqual([{ userId: owner.userId, tier: 'lg' }]);
|
||||
});
|
||||
});
|
||||
73
packages/core/src/box/placement.ts
Normal file
73
packages/core/src/box/placement.ts
Normal file
@@ -0,0 +1,73 @@
|
||||
import z from 'zod';
|
||||
|
||||
import { ErrorCodes, VisibleError } from '../error.js';
|
||||
import { Machine } from '../machine/index.js';
|
||||
import { BoxTier } from './box.sql.js';
|
||||
|
||||
/**
|
||||
* Deciding which host a box runs on.
|
||||
*
|
||||
* This is a seam and not an algorithm. `box.machineId` is set once, when the
|
||||
* box is created, and everything downstream — a session, its job, the state
|
||||
* reports that follow — reaches the right hardware by joining through the box.
|
||||
* So there is exactly one moment where placement happens, and the value of
|
||||
* naming it now is that a real scheduler replaces this file and nothing else.
|
||||
*
|
||||
* The wrong shape, and the tempting one, is to place a box when a *run* is
|
||||
* requested. That spreads the decision across every caller that starts
|
||||
* something and leaves nowhere to put a scheduler later.
|
||||
*/
|
||||
export namespace Placement {
|
||||
export const Request = z.object({
|
||||
userId: z.string().meta({ description: 'Who the box is for' }),
|
||||
tier: z.enum(BoxTier.enumValues).meta({ description: 'The size that was asked for' })
|
||||
});
|
||||
|
||||
export type Request = z.infer<typeof Request>;
|
||||
|
||||
/**
|
||||
* Answers "which host should run this box?" with a machine id.
|
||||
*
|
||||
* Asynchronous and allowed to refuse: capacity is a real answer, and a
|
||||
* placer that cannot honour a request must say so rather than return
|
||||
* something the insert would reject — `box.machineId` is not nullable.
|
||||
*/
|
||||
export type Placer = (request: Request) => Promise<string>;
|
||||
|
||||
/**
|
||||
* The implementation there is hardware for: place it on the caller's host.
|
||||
*
|
||||
* Deliberately refuses when the answer is not forced. With no host there is
|
||||
* nothing to place on; with several there is a choice to make and no policy
|
||||
* to make it with, and picking the first row would be a scheduling decision
|
||||
* taken by accident and impossible to find later. Refusing keeps the choice
|
||||
* in this one function.
|
||||
*/
|
||||
export const onlyHost: Placer = async (request) => {
|
||||
const hosts = await Machine.listByOwner(request.userId);
|
||||
|
||||
if (hosts.length === 0) {
|
||||
throw new VisibleError(
|
||||
'not_found',
|
||||
ErrorCodes.NotFound.RESOURCE_NOT_FOUND,
|
||||
'You have no registered host to run a box on'
|
||||
);
|
||||
}
|
||||
if (hosts.length > 1) {
|
||||
// Not a caller error: the request is fine and the system cannot yet
|
||||
// answer it. An orchestrator is what closes this. todo(d-0048)
|
||||
throw new VisibleError(
|
||||
'internal',
|
||||
ErrorCodes.Server.SERVICE_UNAVAILABLE,
|
||||
'More than one host could run this box, and choosing between them is not supported yet'
|
||||
);
|
||||
}
|
||||
|
||||
return hosts[0]!.id;
|
||||
};
|
||||
|
||||
/** Place a box, using `onlyHost` unless a caller supplies its own placer. */
|
||||
export async function choose(request: Request, placer: Placer = onlyHost): Promise<string> {
|
||||
return placer(Request.parse(request));
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,13 @@
|
||||
import { and, desc, eq, isNull, sql } from 'drizzle-orm';
|
||||
import { and, desc, eq, inArray, isNull, sql } from 'drizzle-orm';
|
||||
import z from 'zod';
|
||||
|
||||
import { BoxTable, BoxTier } from '../box/box.sql.js';
|
||||
import { Box } from '../box/index.js';
|
||||
import { Database } from '../db/index.js';
|
||||
import { ErrorCodes, VisibleError } from '../error.js';
|
||||
import { Examples } from '../examples.js';
|
||||
import { fn } from '../fn.js';
|
||||
import { GameTable } from '../game/game.sql.js';
|
||||
import { SessionState, SessionTable } from './session.sql.js';
|
||||
|
||||
/**
|
||||
@@ -16,6 +20,15 @@ import { SessionState, SessionTable } from './session.sql.js';
|
||||
* value once.
|
||||
*/
|
||||
export namespace Session {
|
||||
/**
|
||||
* One wording for "that box is busy", however it was discovered.
|
||||
*
|
||||
* Both the read in {@link activeForBox} and the unique index behind
|
||||
* {@link request} report it, and a caller must not be able to tell which,
|
||||
* because that would only tell it how close the race was.
|
||||
*/
|
||||
export const BOX_BUSY = 'That box already has a run that has not stopped';
|
||||
|
||||
export const Info = z
|
||||
.object({
|
||||
id: z.string().meta({
|
||||
@@ -39,7 +52,8 @@ export namespace Session {
|
||||
example: Examples.Session.state
|
||||
}),
|
||||
ticket: z.string().nullable().optional().meta({
|
||||
description: 'Current iroh connect ticket, or null before neshub mints one',
|
||||
description:
|
||||
'Current connect ticket. Null before one has been minted, and null again once the run has stopped — a run that is not there has no address',
|
||||
example: Examples.Session.ticket
|
||||
}),
|
||||
timeStarted: z.string().nullable().optional().meta({
|
||||
@@ -81,6 +95,41 @@ export namespace Session {
|
||||
}
|
||||
);
|
||||
|
||||
/** Postgres refusing a second row for the same key. */
|
||||
function isUniqueViolation(err: unknown): boolean {
|
||||
const e = err as { code?: string; cause?: { code?: string } };
|
||||
return e?.code === '23505' || e?.cause?.code === '23505';
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask for a run of a box, and let the database refuse a second one.
|
||||
*
|
||||
* The same argument as {@link compareAndSetState}, one step earlier. A
|
||||
* caller reading {@link activeForBox} first and the insert being refused
|
||||
* are different properties: the read is a nicer error message, the unique
|
||||
* index is the invariant. Two requests that both read "nothing is running"
|
||||
* before either inserts each get a row, and the host is then handed the
|
||||
* same box to start twice — which is exactly what the state claim exists
|
||||
* to prevent.
|
||||
*
|
||||
* The refusal is the same 409 a caller gets from the read, and worded
|
||||
* identically, because from outside they are the same fact and which one
|
||||
* answered is a timing detail.
|
||||
*/
|
||||
export const request = fn(
|
||||
Info.pick({ id: true, boxId: true, gameId: true, linkedAccountId: true }),
|
||||
async (input) => {
|
||||
try {
|
||||
return await create(input);
|
||||
} catch (err) {
|
||||
if (isUniqueViolation(err)) {
|
||||
throw new VisibleError('already_exists', ErrorCodes.Validation.INVALID_STATE, BOX_BUSY);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export const fromID = fn(Info.shape.id, async (id) => {
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
@@ -97,9 +146,11 @@ export namespace Session {
|
||||
/**
|
||||
* The run currently occupying a box, if any.
|
||||
*
|
||||
* Newest first and limited to one: a box has at most one live session by
|
||||
* construction, and if that ever stops being true this is the query that
|
||||
* should start refusing rather than picking a winner silently.
|
||||
* At most one row can match: `session_box_active_unique` is a unique index
|
||||
* on this exact predicate, so "newest first, limited to one" describes the
|
||||
* query and not a choice being made. Reading this before inserting gives a
|
||||
* caller a better message than a constraint violation; it is not what makes
|
||||
* the answer single. See {@link request}.
|
||||
*/
|
||||
export const activeForBox = fn(Info.shape.boxId, async (boxId) => {
|
||||
return Database.use(async (tx) => {
|
||||
@@ -175,7 +226,12 @@ export namespace Session {
|
||||
? { timeStarted: sql`coalesce(${SessionTable.timeStarted}, ${now})` }
|
||||
: {}),
|
||||
...(input.state === 'ended' || input.state === 'failed'
|
||||
? { timeStopped: sql`coalesce(${SessionTable.timeStopped}, ${now})` }
|
||||
? {
|
||||
timeStopped: sql`coalesce(${SessionTable.timeStopped}, ${now})`,
|
||||
// A stopped run has no address, whichever writer
|
||||
// stopped it.
|
||||
ticket: null
|
||||
}
|
||||
: {})
|
||||
})
|
||||
.where(and(eq(SessionTable.id, input.id), isNull(SessionTable.timeDeleted)))
|
||||
@@ -188,6 +244,399 @@ export namespace Session {
|
||||
}
|
||||
);
|
||||
|
||||
/**
|
||||
* The states an agent is allowed to move a run into.
|
||||
*
|
||||
* `requested` is missing on purpose: it is written once, when the row is
|
||||
* created, and nothing may put a run back there.
|
||||
*/
|
||||
export const ReportableState = z.enum(['starting', 'live', 'ended', 'failed']);
|
||||
|
||||
export type ReportableState = z.infer<typeof ReportableState>;
|
||||
|
||||
/**
|
||||
* Where a run may go next, and nowhere else.
|
||||
*
|
||||
* `requested → live` is missing although it is the tempting shortcut:
|
||||
* skipping `starting` means nothing ever holds the claim, and the claim is
|
||||
* the only mutual exclusion in this design. `ended` and `failed` are
|
||||
* terminal, so their entries are empty rather than absent — a state with no
|
||||
* exits is a fact worth writing down.
|
||||
*/
|
||||
export const NEXT_STATES: Record<Info['state'], readonly Info['state'][]> = {
|
||||
requested: ['starting'],
|
||||
starting: ['live', 'failed'],
|
||||
live: ['ended', 'failed'],
|
||||
ended: [],
|
||||
failed: []
|
||||
};
|
||||
|
||||
/**
|
||||
* A unit of work handed to the agent that will carry it out.
|
||||
*
|
||||
* There is no queue: a run in state `requested` *is* the work order, and
|
||||
* the agent that fulfils it moves that same row along. Two sources of truth
|
||||
* for one piece of work is how a queue and a database come to disagree
|
||||
* about whether something ran.
|
||||
*
|
||||
* `kind` is on the wire while there is only one value, so that a second
|
||||
* kind is an addition rather than a redesign of the poll.
|
||||
*/
|
||||
export const Job = z
|
||||
.object({
|
||||
kind: z.literal('session.start').meta({
|
||||
description: 'What the agent is being asked to do',
|
||||
example: 'session.start'
|
||||
}),
|
||||
sessionId: z.string().meta({
|
||||
description: 'The run to report progress against',
|
||||
example: Examples.Session.id
|
||||
}),
|
||||
boxId: z.string().meta({
|
||||
description: 'The box to start',
|
||||
example: Examples.Session.boxId
|
||||
}),
|
||||
boxTier: z.enum(BoxTier.enumValues).meta({
|
||||
description: 'The size the box was asked for, which also sets output geometry',
|
||||
example: Examples.Box.tier
|
||||
}),
|
||||
gameId: z.string().meta({
|
||||
description: 'The game to launch',
|
||||
example: Examples.Session.gameId
|
||||
}),
|
||||
steamAppId: z.number().int().meta({
|
||||
description: 'The same game, in the id the store knows it by',
|
||||
example: Examples.Game.steamAppId
|
||||
}),
|
||||
linkedAccountId: z.string().meta({
|
||||
description: 'Which linked account is playing',
|
||||
example: Examples.Session.linkedAccountId
|
||||
})
|
||||
})
|
||||
.meta({
|
||||
ref: 'SessionJob',
|
||||
description: 'One run waiting to be started, as handed to the agent that will start it'
|
||||
});
|
||||
|
||||
export type Job = z.infer<typeof Job>;
|
||||
|
||||
/**
|
||||
* The work waiting for one host.
|
||||
*
|
||||
* The scope is the join and not a filter the caller asks for: a box names
|
||||
* the hardware it is placed on, a run reaches its hardware through its box,
|
||||
* and so what one set of long-lived credentials can see is decided by this
|
||||
* `where` clause rather than by whoever is holding them.
|
||||
*/
|
||||
export const listJobsForMachine = fn(z.string(), async (machineId) => {
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
.select({ session: SessionTable, box: BoxTable, game: GameTable })
|
||||
.from(SessionTable)
|
||||
.innerJoin(BoxTable, eq(SessionTable.boxId, BoxTable.id))
|
||||
.innerJoin(GameTable, eq(SessionTable.gameId, GameTable.id))
|
||||
.where(
|
||||
and(
|
||||
eq(BoxTable.machineId, machineId),
|
||||
eq(SessionTable.state, 'requested'),
|
||||
isNull(SessionTable.timeDeleted),
|
||||
isNull(BoxTable.timeDeleted)
|
||||
)
|
||||
)
|
||||
.orderBy(SessionTable.timeCreated)
|
||||
.then((rows) =>
|
||||
rows.map(
|
||||
(row): Job => ({
|
||||
kind: 'session.start',
|
||||
sessionId: row.session.id,
|
||||
boxId: row.box.id,
|
||||
boxTier: row.box.tier as Job['boxTier'],
|
||||
gameId: row.game.id,
|
||||
steamAppId: row.game.steamAppId,
|
||||
linkedAccountId: row.session.linkedAccountId
|
||||
})
|
||||
)
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
/** One run, visible only to the host its box is placed on. */
|
||||
export const forMachine = fn(
|
||||
z.object({ id: Info.shape.id, machineId: z.string() }),
|
||||
async (input) => {
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
.select({ session: SessionTable })
|
||||
.from(SessionTable)
|
||||
.innerJoin(BoxTable, eq(SessionTable.boxId, BoxTable.id))
|
||||
.where(
|
||||
and(
|
||||
eq(SessionTable.id, input.id),
|
||||
eq(BoxTable.machineId, input.machineId),
|
||||
isNull(SessionTable.timeDeleted),
|
||||
isNull(BoxTable.timeDeleted)
|
||||
)
|
||||
)
|
||||
.then((rows) => {
|
||||
const row = rows.at(0);
|
||||
return row ? serialize(row.session) : null;
|
||||
});
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
/** One run, visible only to the person who owns its box. */
|
||||
export const forOwner = fn(z.object({ id: Info.shape.id, userId: z.string() }), async (input) => {
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
.select({ session: SessionTable })
|
||||
.from(SessionTable)
|
||||
.innerJoin(BoxTable, eq(SessionTable.boxId, BoxTable.id))
|
||||
.where(
|
||||
and(
|
||||
eq(SessionTable.id, input.id),
|
||||
eq(BoxTable.userId, input.userId),
|
||||
isNull(SessionTable.timeDeleted),
|
||||
isNull(BoxTable.timeDeleted)
|
||||
)
|
||||
)
|
||||
.then((rows) => {
|
||||
const row = rows.at(0);
|
||||
return row ? serialize(row.session) : null;
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
/** The boxes one host is responsible for, as a subquery to scope a write. */
|
||||
function boxesOn(tx: Parameters<Parameters<typeof Database.use>[0]>[0], machineId: string) {
|
||||
return tx
|
||||
.select({ id: BoxTable.id })
|
||||
.from(BoxTable)
|
||||
.where(and(eq(BoxTable.machineId, machineId), isNull(BoxTable.timeDeleted)));
|
||||
}
|
||||
|
||||
/**
|
||||
* Move a run from one exact state to another, or do nothing at all.
|
||||
*
|
||||
* This is the claim, and it is why `setState` is not enough on its own:
|
||||
* updating on the id alone means two agents polling the same work both
|
||||
* succeed and both start the same box. The current state is part of the
|
||||
* `where` clause, so the database decides the winner and the loser gets
|
||||
* null rather than a row. There is one host today, which is exactly why
|
||||
* this would otherwise be built wrong and stay wrong.
|
||||
*
|
||||
* The host is in the same `where` clause. The caller checking first is not
|
||||
* the same thing as the write being scoped, and only one of the two is
|
||||
* still true when somebody adds a second caller.
|
||||
*/
|
||||
export const compareAndSetState = fn(
|
||||
z.object({
|
||||
id: Info.shape.id,
|
||||
machineId: z.string(),
|
||||
from: z.enum(SessionState.enumValues),
|
||||
to: z.enum(SessionState.enumValues),
|
||||
errorMessage: Info.shape.errorMessage
|
||||
}),
|
||||
async (input) => {
|
||||
const now = sql`now()`;
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
.update(SessionTable)
|
||||
.set({
|
||||
state: input.to,
|
||||
errorMessage: input.to === 'failed' ? (input.errorMessage ?? null) : null,
|
||||
...(input.to === 'live'
|
||||
? { timeStarted: sql`coalesce(${SessionTable.timeStarted}, ${now})` }
|
||||
: {}),
|
||||
...(input.to === 'ended' || input.to === 'failed'
|
||||
? {
|
||||
timeStopped: sql`coalesce(${SessionTable.timeStopped}, ${now})`,
|
||||
// A stopped run has no address. Publishing one is
|
||||
// already refused, so keeping the last one would
|
||||
// leave the only readable ticket for a dead run
|
||||
// being the one nothing may replace — and a client
|
||||
// that polls would dial it.
|
||||
ticket: null
|
||||
}
|
||||
: {})
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(SessionTable.id, input.id),
|
||||
eq(SessionTable.state, input.from),
|
||||
isNull(SessionTable.timeDeleted),
|
||||
inArray(SessionTable.boxId, boxesOn(tx, input.machineId))
|
||||
)
|
||||
)
|
||||
.returning()
|
||||
.then((rows) => {
|
||||
const row = rows.at(0);
|
||||
return row ? serialize(row) : null;
|
||||
});
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
/**
|
||||
* What happened when an agent reported a state.
|
||||
*
|
||||
* Four outcomes that look alike from a distance and are not, which is the
|
||||
* whole reason this is not a boolean:
|
||||
*
|
||||
* - `forbidden` — no such run, or it is not on this host. One answer for
|
||||
* both, so reporting states at ids cannot be used to discover them.
|
||||
* - `unchanged` — already in that state. A retry after a lost response is
|
||||
* not a broken agent and must not be told it is.
|
||||
* - `illegal` — not a transition that exists. The row does not move.
|
||||
* - `lost` — a legal transition that something else got to first.
|
||||
* - `moved` — it happened.
|
||||
*/
|
||||
export type TransitionOutcome = 'forbidden' | 'unchanged' | 'illegal' | 'lost' | 'moved';
|
||||
|
||||
export interface TransitionResult {
|
||||
outcome: TransitionOutcome;
|
||||
session: Info | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* What a run reaching a state means for the box underneath it.
|
||||
*
|
||||
* The box has its own three states and nothing was writing them, so a box
|
||||
* read `created` while a run on it was `live` — the screens that show a
|
||||
* person what their hardware is doing would all have been wrong. The two
|
||||
* state machines are not the same shape and should not be: a box has no
|
||||
* `starting`, deliberately, because that transition is synchronous from
|
||||
* the agent's side and a state nobody sets is a state that lies. So only
|
||||
* the states that mean something to the box are mapped, and `requested`
|
||||
* and `starting` map to nothing at all.
|
||||
*
|
||||
* `failed` is a `stopped` box that did not stop cleanly, which is the
|
||||
* distinction `stopClean` exists for: "it is not running" and "it faulted"
|
||||
* are different facts and the difference lives in the reason.
|
||||
*/
|
||||
function boxStateFor(
|
||||
run: Info
|
||||
): { state: 'running' | 'stopped'; stopReason: string | null; stopClean: boolean | null } | null {
|
||||
switch (run.state) {
|
||||
case 'live':
|
||||
return { state: 'running', stopReason: null, stopClean: null };
|
||||
case 'ended':
|
||||
return { state: 'stopped', stopReason: null, stopClean: true };
|
||||
case 'failed':
|
||||
// The run's own reason, so a box explains its stop in the words
|
||||
// the agent used rather than in a second wording of one event.
|
||||
return { state: 'stopped', stopReason: run.errorMessage ?? null, stopClean: false };
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export const transition = fn(
|
||||
z.object({
|
||||
id: Info.shape.id,
|
||||
machineId: z.string(),
|
||||
state: z.enum(SessionState.enumValues),
|
||||
errorMessage: Info.shape.errorMessage
|
||||
}),
|
||||
async (input): Promise<TransitionResult> => {
|
||||
// One transaction, because "this run is live" and "the box under it
|
||||
// is running" are one fact written to two tables. Committing the
|
||||
// first without the second is how a box gets stuck `running` with
|
||||
// nothing running on it, and nothing here would ever correct it.
|
||||
return Database.transaction(async (): Promise<TransitionResult> => {
|
||||
const current = await forMachine({ id: input.id, machineId: input.machineId });
|
||||
if (!current) return { outcome: 'forbidden', session: null };
|
||||
if (current.state === input.state) return { outcome: 'unchanged', session: current };
|
||||
if (!NEXT_STATES[current.state].includes(input.state)) {
|
||||
return { outcome: 'illegal', session: current };
|
||||
}
|
||||
|
||||
const moved = await compareAndSetState({
|
||||
id: input.id,
|
||||
machineId: input.machineId,
|
||||
from: current.state,
|
||||
to: input.state,
|
||||
errorMessage: input.errorMessage
|
||||
});
|
||||
// The state read above is not the state written below, and the
|
||||
// gap is where two agents race. Nothing moved means somebody
|
||||
// else did — and then the box is that caller's to update, not
|
||||
// this one's.
|
||||
if (!moved) return { outcome: 'lost', session: current };
|
||||
|
||||
const box = boxStateFor(moved);
|
||||
if (box) {
|
||||
await Box.setState({ id: moved.boxId, ...box });
|
||||
}
|
||||
|
||||
return { outcome: 'moved', session: moved };
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
export interface TicketResult {
|
||||
outcome: 'forbidden' | 'unclaimed' | 'closed' | 'published';
|
||||
session: Info | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The states a run can have an address in.
|
||||
*
|
||||
* `starting` is in and `requested` is out, which is the whole distinction:
|
||||
* a ticket is the address of something being brought up, so publishing one
|
||||
* means the host has taken the work. It cannot have an address for a run it
|
||||
* has not claimed, and the terminal states are out because a run that is
|
||||
* not there has no address at all.
|
||||
*/
|
||||
const ADDRESSABLE = ['starting', 'live'] as const;
|
||||
|
||||
/**
|
||||
* Publish a ticket for a run, on behalf of the host it is placed on.
|
||||
*
|
||||
* A ticket may appear while the state is still `starting` — it is
|
||||
* republished as addresses are discovered, so the client polls and re-reads
|
||||
* rather than keeping the first one. Outside {@link ADDRESSABLE} it is
|
||||
* refused, and the two refusals are separate answers because they are
|
||||
* different mistakes: a run not yet claimed is an agent that skipped a
|
||||
* step, and a run that stopped is one that has nothing left to reach.
|
||||
*/
|
||||
export const publishTicket = fn(
|
||||
z.object({
|
||||
id: Info.shape.id,
|
||||
machineId: z.string(),
|
||||
ticket: z.string().min(1)
|
||||
}),
|
||||
async (input): Promise<TicketResult> => {
|
||||
const current = await forMachine({ id: input.id, machineId: input.machineId });
|
||||
if (!current) return { outcome: 'forbidden', session: null };
|
||||
if (current.state === 'requested') return { outcome: 'unclaimed', session: current };
|
||||
|
||||
return Database.use(async (tx) => {
|
||||
return tx
|
||||
.update(SessionTable)
|
||||
.set({ ticket: input.ticket })
|
||||
.where(
|
||||
and(
|
||||
eq(SessionTable.id, input.id),
|
||||
// The state is in the write and not only in the check
|
||||
// above it, so a run that stops underneath this call
|
||||
// does not acquire an address on the way out.
|
||||
inArray(SessionTable.state, [...ADDRESSABLE]),
|
||||
isNull(SessionTable.timeDeleted),
|
||||
inArray(SessionTable.boxId, boxesOn(tx, input.machineId))
|
||||
)
|
||||
)
|
||||
.returning()
|
||||
.then((rows): TicketResult => {
|
||||
const row = rows.at(0);
|
||||
return row
|
||||
? { outcome: 'published', session: serialize(row) }
|
||||
: { outcome: 'closed', session: current };
|
||||
});
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
export function serialize(input: typeof SessionTable.$inferSelect): z.infer<typeof Info> {
|
||||
return {
|
||||
id: input.id,
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { index, pgEnum, pgTable, text } from 'drizzle-orm/pg-core';
|
||||
import { sql } from 'drizzle-orm';
|
||||
import { index, pgEnum, pgTable, text, uniqueIndex } from 'drizzle-orm/pg-core';
|
||||
|
||||
import { id, timestamps, ulid, utc } from '../db/types.js';
|
||||
import { BoxTable } from '../box/box.sql.js';
|
||||
import { id, timestamps, ulid, utc } from '../db/types.js';
|
||||
import { GameTable } from '../game/game.sql.js';
|
||||
import { LinkedAccountTable } from '../user/linked-account.sql.js';
|
||||
|
||||
@@ -66,6 +67,18 @@ export const SessionTable = pgTable(
|
||||
(t) => [
|
||||
index('session_box_idx').on(t.boxId),
|
||||
index('session_state_idx').on(t.state),
|
||||
// A box runs one thing at a time, and this is where that is true.
|
||||
//
|
||||
// The caller checking first is a nicer error message; it is not the
|
||||
// invariant. Two requests that both read "nothing is running" before
|
||||
// either inserts each get a row, and the host is then handed the same
|
||||
// box to start twice — the identical failure the state claim exists to
|
||||
// prevent, one step earlier. So the predicate is exactly the one
|
||||
// `Session.activeForBox` asks about, and the database refuses the
|
||||
// second row rather than a caller remembering to.
|
||||
uniqueIndex('session_box_active_unique')
|
||||
.on(t.boxId)
|
||||
.where(sql`time_stopped is null and time_deleted is null`),
|
||||
// Metering reads "sessions in this window", and this table is what
|
||||
// billing sums, so the time index is not speculative. ref(d-0048)
|
||||
index('session_started_idx').on(t.timeStarted)
|
||||
|
||||
@@ -41,7 +41,7 @@ async function scene(label: string, steamAppId: number) {
|
||||
label,
|
||||
tier: 'sm'
|
||||
});
|
||||
return { owner, box, gameId: await newGame(steamAppId) };
|
||||
return { owner, machineId, box, gameId: await newGame(steamAppId) };
|
||||
}
|
||||
|
||||
afterAll(async () => {
|
||||
@@ -58,6 +58,18 @@ afterAll(async () => {
|
||||
}
|
||||
});
|
||||
|
||||
/** A scene with the run already requested. */
|
||||
async function requestedRun(label: string, steamAppId: number) {
|
||||
const s = await scene(label, steamAppId);
|
||||
const session = await Session.request({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
});
|
||||
return { ...s, session };
|
||||
}
|
||||
|
||||
describe('Session', () => {
|
||||
test('a session starts requested, with no ticket and no times', async () => {
|
||||
const { owner, box, gameId } = await scene('ses-defaults', 5400);
|
||||
@@ -173,3 +185,456 @@ describe('Session', () => {
|
||||
expect(await Session.listByBox(box.id)).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session jobs', () => {
|
||||
test('a requested session is the job, and it carries its kind', async () => {
|
||||
const { owner, machineId, box, gameId } = await scene('ses-job-kind', 5410);
|
||||
const session = await Session.create({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: box.id,
|
||||
gameId,
|
||||
linkedAccountId: owner.linkedAccountId
|
||||
});
|
||||
|
||||
const jobs = await Session.listJobsForMachine(machineId);
|
||||
expect(jobs).toHaveLength(1);
|
||||
// The kind is on the wire from the first day there is only one, so the
|
||||
// second kind is an addition rather than a redesign.
|
||||
expect(jobs[0]!.kind).toBe('session.start');
|
||||
expect(jobs[0]!.sessionId).toBe(session.id);
|
||||
expect(jobs[0]!.boxId).toBe(box.id);
|
||||
expect(jobs[0]!.boxTier).toBe('sm');
|
||||
expect(jobs[0]!.gameId).toBe(gameId);
|
||||
expect(jobs[0]!.steamAppId).toBe(5410);
|
||||
expect(jobs[0]!.linkedAccountId).toBe(owner.linkedAccountId);
|
||||
});
|
||||
|
||||
test('a job belongs to the machine its box is placed on and to no other', async () => {
|
||||
const mine = await scene('ses-job-mine', 5411);
|
||||
const theirs = await scene('ses-job-theirs', 5412);
|
||||
|
||||
const session = await Session.create({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: theirs.box.id,
|
||||
gameId: theirs.gameId,
|
||||
linkedAccountId: theirs.owner.linkedAccountId
|
||||
});
|
||||
|
||||
// The scope is the join, not a filter the caller asks for. A machine
|
||||
// credential is a long-lived secret on hardware in somebody's home, so
|
||||
// what one leaking can reach is decided here.
|
||||
expect(await Session.listJobsForMachine(mine.machineId)).toHaveLength(0);
|
||||
expect((await Session.listJobsForMachine(theirs.machineId)).map((j) => j.sessionId)).toEqual([
|
||||
session.id
|
||||
]);
|
||||
});
|
||||
|
||||
test('only a requested session is work; a claimed one is not offered again', async () => {
|
||||
const { owner, machineId, box, gameId } = await scene('ses-job-claimed', 5413);
|
||||
const session = await Session.create({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: box.id,
|
||||
gameId,
|
||||
linkedAccountId: owner.linkedAccountId
|
||||
});
|
||||
|
||||
expect(await Session.listJobsForMachine(machineId)).toHaveLength(1);
|
||||
await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(await Session.listJobsForMachine(machineId)).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session claim', () => {
|
||||
async function requested(label: string, steamAppId: number) {
|
||||
const s = await scene(label, steamAppId);
|
||||
const session = await Session.create({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: s.box.id,
|
||||
gameId: s.gameId,
|
||||
linkedAccountId: s.owner.linkedAccountId
|
||||
});
|
||||
return { ...s, session };
|
||||
}
|
||||
|
||||
test('the claim is a compare-and-set, so the second attempt finds nothing to move', async () => {
|
||||
const { machineId, session } = await requested('ses-cas', 5420);
|
||||
|
||||
const won = await Session.compareAndSetState({
|
||||
id: session.id,
|
||||
machineId,
|
||||
from: 'requested',
|
||||
to: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(won?.state).toBe('starting');
|
||||
|
||||
// The same attempt again. The row is no longer `requested`, so the
|
||||
// update matches nothing — which is what stops two agents from both
|
||||
// starting the same box. Updating on the id alone would succeed twice.
|
||||
const lost = await Session.compareAndSetState({
|
||||
id: session.id,
|
||||
machineId,
|
||||
from: 'requested',
|
||||
to: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(lost).toBeNull();
|
||||
expect((await Session.fromID(session.id))?.state).toBe('starting');
|
||||
});
|
||||
|
||||
test('a machine that is not the box’s host cannot move the row', async () => {
|
||||
const { session } = await requested('ses-cas-mine', 5421);
|
||||
const other = await scene('ses-cas-other', 5422);
|
||||
|
||||
const result = await Session.transition({
|
||||
id: session.id,
|
||||
machineId: other.machineId,
|
||||
state: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(result.outcome).toBe('forbidden');
|
||||
expect((await Session.fromID(session.id))?.state).toBe('requested');
|
||||
|
||||
// And the compare-and-set is scoped in the same query, not only by the
|
||||
// classification above it.
|
||||
expect(
|
||||
await Session.compareAndSetState({
|
||||
id: session.id,
|
||||
machineId: other.machineId,
|
||||
from: 'requested',
|
||||
to: 'starting',
|
||||
errorMessage: null
|
||||
})
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
test('a session that does not exist is refused the same way as one that is not yours', async () => {
|
||||
const other = await scene('ses-cas-ghost', 5423);
|
||||
const result = await Session.transition({
|
||||
// A well-formed id for a row that was never written.
|
||||
id: Identifier.ascending('session'),
|
||||
machineId: other.machineId,
|
||||
state: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
// Same answer as somebody else's session: an agent must not be able to
|
||||
// learn which ids exist by reporting states at them.
|
||||
expect(result.outcome).toBe('forbidden');
|
||||
});
|
||||
|
||||
test('re-reporting the state you already reported changes nothing', async () => {
|
||||
const { machineId, session } = await requested('ses-repeat', 5424);
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
const again = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
// A retry after a lost response is not a broken agent.
|
||||
expect(again.outcome).toBe('unchanged');
|
||||
expect(again.session?.state).toBe('starting');
|
||||
});
|
||||
|
||||
test('a transition off the table is refused and the row does not move', async () => {
|
||||
const { machineId, session } = await requested('ses-illegal', 5425);
|
||||
|
||||
// Skipping `starting` means nothing ever holds the claim, and the claim
|
||||
// is the only mutual exclusion here — so it is refused however tempting
|
||||
// the shortcut looks.
|
||||
const skipped = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'live',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(skipped.outcome).toBe('illegal');
|
||||
expect((await Session.fromID(session.id))?.state).toBe('requested');
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'failed', errorMessage: 'no' });
|
||||
|
||||
// Terminal is terminal: a dead session cannot be resurrected.
|
||||
const raised = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'live',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(raised.outcome).toBe('illegal');
|
||||
expect((await Session.fromID(session.id))?.state).toBe('failed');
|
||||
});
|
||||
|
||||
test('the timestamps survive a duplicate report, which is what billing rests on', async () => {
|
||||
const { machineId, session } = await requested('ses-idempotent', 5426);
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
const live = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'live',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(live.session?.timeStarted).not.toBeNull();
|
||||
|
||||
const repeat = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'live',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(repeat.session?.timeStarted).toBe(live.session!.timeStarted);
|
||||
});
|
||||
|
||||
test('publishing a ticket is scoped to the host too', async () => {
|
||||
const { machineId, session } = await requested('ses-ticket-scope', 5427);
|
||||
const other = await scene('ses-ticket-other', 5428);
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
|
||||
const refused = await Session.publishTicket({
|
||||
id: session.id,
|
||||
machineId: other.machineId,
|
||||
ticket: 'stolen'
|
||||
});
|
||||
expect(refused.outcome).toBe('forbidden');
|
||||
expect((await Session.fromID(session.id))?.ticket).toBeNull();
|
||||
|
||||
// A ticket may appear while the state is still `starting`.
|
||||
const first = await Session.publishTicket({ id: session.id, machineId, ticket: 'one' });
|
||||
expect(first.outcome).toBe('published');
|
||||
expect(first.session?.ticket).toBe('one');
|
||||
expect(first.session?.state).toBe('starting');
|
||||
|
||||
const second = await Session.publishTicket({ id: session.id, machineId, ticket: 'two' });
|
||||
expect(second.session?.ticket).toBe('two');
|
||||
});
|
||||
|
||||
test('a stopped session has no address to publish', async () => {
|
||||
const { machineId, session } = await requested('ses-ticket-dead', 5429);
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'live', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'ended', errorMessage: null });
|
||||
|
||||
const result = await Session.publishTicket({ id: session.id, machineId, ticket: 'late' });
|
||||
expect(result.outcome).toBe('closed');
|
||||
expect((await Session.fromID(session.id))?.ticket).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session one active run per box', () => {
|
||||
test('the database refuses a second unstopped run, not just the caller', async () => {
|
||||
const { owner, machineId, box, gameId } = await scene('ses-one-active', 5450);
|
||||
const mk = () =>
|
||||
Session.request({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: box.id,
|
||||
gameId,
|
||||
linkedAccountId: owner.linkedAccountId
|
||||
});
|
||||
|
||||
const first = await mk();
|
||||
expect(first.state).toBe('requested');
|
||||
|
||||
// The interleaving `POST /session` permits: both callers read
|
||||
// `activeForBox` and see nothing, then both insert. The read is a
|
||||
// message; the unique index is the invariant, so the second insert is
|
||||
// refused rather than producing a row.
|
||||
const before = await Session.activeForBox(box.id);
|
||||
expect(before?.id).toBe(first.id);
|
||||
await expect(mk()).rejects.toThrow(Session.BOX_BUSY);
|
||||
|
||||
expect(await Session.listByBox(box.id)).toHaveLength(1);
|
||||
// The point of all of it: the host is handed one launch, not two.
|
||||
expect(await Session.listJobsForMachine(machineId)).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('a box that has stopped running is free to run again', async () => {
|
||||
const { owner, machineId, box, gameId } = await scene('ses-one-active-reuse', 5451);
|
||||
const mk = () =>
|
||||
Session.request({
|
||||
id: Identifier.ascending('session'),
|
||||
boxId: box.id,
|
||||
gameId,
|
||||
linkedAccountId: owner.linkedAccountId
|
||||
});
|
||||
|
||||
const first = await mk();
|
||||
await Session.transition({ id: first.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({ id: first.id, machineId, state: 'live', errorMessage: null });
|
||||
await Session.transition({ id: first.id, machineId, state: 'ended', errorMessage: null });
|
||||
|
||||
// The index is partial for exactly this reason: a box is a durable
|
||||
// thing and playing twice is the ordinary case, so a stopped run must
|
||||
// not occupy the slot forever.
|
||||
const second = await mk();
|
||||
expect(second.id).not.toBe(first.id);
|
||||
expect(await Session.listByBox(box.id)).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session tickets and the end of a run', () => {
|
||||
test('stopping a run takes its address away', async () => {
|
||||
const { machineId, session } = await requestedRun('ses-ticket-cleared', 5452);
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'live', errorMessage: null });
|
||||
expect(
|
||||
(await Session.publishTicket({ id: session.id, machineId, ticket: 'live-address' })).session
|
||||
?.ticket
|
||||
).toBe('live-address');
|
||||
|
||||
const ended = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'ended',
|
||||
errorMessage: null
|
||||
});
|
||||
// Publishing an address for a stopped run is already refused, so a
|
||||
// kept one would be the only ticket a client can read for a dead run
|
||||
// and the one nothing is allowed to replace. It would be dialled.
|
||||
expect(ended.outcome).toBe('moved');
|
||||
expect(ended.session?.ticket).toBeNull();
|
||||
expect((await Session.fromID(session.id))?.ticket).toBeNull();
|
||||
});
|
||||
|
||||
test('a run that failed does not keep an address either', async () => {
|
||||
const { machineId, session } = await requestedRun('ses-ticket-cleared-fail', 5453);
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.publishTicket({ id: session.id, machineId, ticket: 'starting-address' });
|
||||
|
||||
const failed = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'failed',
|
||||
errorMessage: 'the guest never came up'
|
||||
});
|
||||
expect(failed.session?.ticket).toBeNull();
|
||||
// The reason survives; only the address goes.
|
||||
expect(failed.session?.errorMessage).toBe('the guest never came up');
|
||||
});
|
||||
|
||||
test('the unscoped primitive clears it too, whichever writer stops a run', async () => {
|
||||
const { session } = await requestedRun('ses-ticket-cleared-setstate', 5454);
|
||||
await Session.setTicket({ id: session.id, ticket: 'an-address' });
|
||||
|
||||
const ended = await Session.setState({
|
||||
id: session.id,
|
||||
state: 'ended',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(ended?.ticket).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session and the box underneath it', () => {
|
||||
test('a live run is what makes its box running', async () => {
|
||||
const { machineId, box, session } = await requestedRun('ses-box-live', 5460);
|
||||
// A box starts out `created` and nothing had ever moved it, so it read
|
||||
// `created` while a run on it was `live`.
|
||||
expect((await Box.fromID(box.id))?.state).toBe('created');
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
// `starting` is deliberately not a box state: that transition is
|
||||
// synchronous from the agent's side, so nothing would ever write it.
|
||||
expect((await Box.fromID(box.id))?.state).toBe('created');
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'live', errorMessage: null });
|
||||
const running = await Box.fromID(box.id);
|
||||
expect(running?.state).toBe('running');
|
||||
expect(running?.stopReason).toBeNull();
|
||||
expect(running?.stopClean).toBeNull();
|
||||
});
|
||||
|
||||
test('a run that ends stops its box, cleanly', async () => {
|
||||
const { machineId, box, session } = await requestedRun('ses-box-ended', 5461);
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'live', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'ended', errorMessage: null });
|
||||
|
||||
const stopped = await Box.fromID(box.id);
|
||||
expect(stopped?.state).toBe('stopped');
|
||||
expect(stopped?.stopClean).toBe(true);
|
||||
expect(stopped?.stopReason).toBeNull();
|
||||
});
|
||||
|
||||
test('a run that fails stops its box in the words the agent used', async () => {
|
||||
const { machineId, box, session } = await requestedRun('ses-box-failed', 5462);
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'failed',
|
||||
errorMessage: 'the guest never came up'
|
||||
});
|
||||
|
||||
const stopped = await Box.fromID(box.id);
|
||||
expect(stopped?.state).toBe('stopped');
|
||||
// "It is not running" and "it faulted" are different facts, and the
|
||||
// difference lives in the reason rather than in a fourth state.
|
||||
expect(stopped?.stopClean).toBe(false);
|
||||
expect(stopped?.stopReason).toBe('the guest never came up');
|
||||
});
|
||||
|
||||
test('a refused report leaves the box alone', async () => {
|
||||
const { machineId, box, session } = await requestedRun('ses-box-untouched', 5463);
|
||||
const other = await scene('ses-box-otherhost', 5464);
|
||||
|
||||
const refused = await Session.transition({
|
||||
id: session.id,
|
||||
machineId: other.machineId,
|
||||
state: 'starting',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(refused.outcome).toBe('forbidden');
|
||||
|
||||
// An illegal transition does not move the run, so it must not move the
|
||||
// box either — otherwise the box records a run that never happened.
|
||||
const illegal = await Session.transition({
|
||||
id: session.id,
|
||||
machineId,
|
||||
state: 'live',
|
||||
errorMessage: null
|
||||
});
|
||||
expect(illegal.outcome).toBe('illegal');
|
||||
expect((await Box.fromID(box.id))?.state).toBe('created');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Session tickets need a claim first', () => {
|
||||
test('a run nobody has claimed has no address to publish', async () => {
|
||||
const { machineId, session } = await requestedRun('ses-ticket-unclaimed', 5465);
|
||||
|
||||
// A ticket is the address of something being brought up, so publishing
|
||||
// one for a `requested` run means the agent skipped the claim — the
|
||||
// step that is the only mutual exclusion in the design.
|
||||
const early = await Session.publishTicket({ id: session.id, machineId, ticket: 'too-soon' });
|
||||
expect(early.outcome).toBe('unclaimed');
|
||||
expect((await Session.fromID(session.id))?.ticket).toBeNull();
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
const now = await Session.publishTicket({ id: session.id, machineId, ticket: 'in-time' });
|
||||
expect(now.outcome).toBe('published');
|
||||
expect(now.session?.ticket).toBe('in-time');
|
||||
});
|
||||
|
||||
test('the two refusals are different answers, because they are different mistakes', async () => {
|
||||
const { machineId, session } = await requestedRun('ses-ticket-refusals', 5466);
|
||||
const unclaimed = await Session.publishTicket({ id: session.id, machineId, ticket: 'a' });
|
||||
|
||||
await Session.transition({ id: session.id, machineId, state: 'starting', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'live', errorMessage: null });
|
||||
await Session.transition({ id: session.id, machineId, state: 'ended', errorMessage: null });
|
||||
const closed = await Session.publishTicket({ id: session.id, machineId, ticket: 'b' });
|
||||
|
||||
// One is an agent that has not claimed the work; the other is a run
|
||||
// with nothing left to reach. Collapsing them would tell an agent
|
||||
// retrying the wrong thing.
|
||||
expect(unclaimed.outcome).toBe('unclaimed');
|
||||
expect(closed.outcome).toBe('closed');
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user