From 77d4782c86d5cddbc7274ffb771e7fc7bf3c9842 Mon Sep 17 00:00:00 2001 From: Wanjohi Date: Wed, 26 Aug 2026 18:23:42 +0300 Subject: [PATCH] docs: split CLAUDE.md by scope, and say what this repo is MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLAUDE.md was 1,324 lines and all of it was about the TypeScript half, written before there was another half. Every line of it loaded on every turn regardless of what was being worked on, which is a real cost paid constantly for context that is usually irrelevant. Split by where it applies, so each guide loads when you are in the directory it describes: packages/core/CLAUDE.md 694 domain modules, fn(), actor, errors, auth apps/api/CLAUDE.md 284 routes, registration, error flow docs/alchemy.md 345 stages, bindings, secrets, the CLI CLAUDE.md 72 the repo, both toolchains, two hard rules Nothing was rewritten or dropped — the three files are the original text, verified identical after the split. What the root file now carries is only what is true repo-wide: the layout, the commands, where the detail lives, and the two rules that are not style preferences. One of those is that nothing closed may enter this repo, which is here because it has already been caught once. The README described a streaming platform in four bullets and did not mention that half the repository is Rust that runs inside a virtual machine. It now says what each component does, why a micro-VM rather than a container, what is deliberately absent, and what decides whether a thing is open — data is, capacity is not. It also says plainly that this is mid-rewrite and the docs are behind. Someone arriving at a repo whose documentation does not match its tree should be told that by the README rather than discover it. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 1358 ++------------------------------------- README.md | 84 ++- apps/api/CLAUDE.md | 284 ++++++++ docs/alchemy.md | 345 ++++++++++ packages/core/CLAUDE.md | 694 ++++++++++++++++++++ 5 files changed, 1455 insertions(+), 1310 deletions(-) create mode 100644 apps/api/CLAUDE.md create mode 100644 docs/alchemy.md create mode 100644 packages/core/CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md index e5ff2e84..d6a00bb3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,1324 +1,72 @@ -# @nestri/core — Domain Module Guide +# nestri -## Structure +Open-source cloud gaming: a control plane and the guest components that run +inside a box. **This repository is public.** That is the single most important +fact about it and most of the rules below follow from it. -Every domain module lives in `packages/core/src//` as either a top-level namespace or a nested sub-module: +## Layout + +Split by *what a thing is*, not by what language it is written in. ``` -src// - ├── .sql.ts # (optional) Drizzle table for the parent entity - ├── index.ts # Parent namespace (e.g. User, Game, Team) - ├── .sql.ts # Sub-module table (e.g. fingerprint.sql.ts) - └── .ts # Sub-module namespace (e.g. export namespace Fingerprint) +apps/ what runs api, auth (TS) · nescope, neswire, nescapture (Rust) +crates/ shared Rust nesprotocol +packages/ shared TS core, auth +docs/ long-form alchemy.md ``` -### Sub-modules nested under parents +Both toolchains live at the root: `package.json` is the Bun workspace, +`Cargo.toml` the Cargo one. -| File | Namespace | Why | -| ----------------------- | --------------- | ------------------------------------- | -| `user/linked-account.*` | `LinkedAccount` | A user's OAuth/gaming identities | -| `user/fingerprint.*` | `Fingerprint` | SSH key fingerprints | -| `game/download.*` | `GameDownload` | Per-host game depot downloads | -| `user/library.*` | `Library` | User's owned games with playtime | -| `team/member.*` | `Member` | Team membership with role | -| `game/depot.*` | `Depot` | Platform-specific game content depots | +| | | +|---|---| +| `bun install` | dependencies | +| `bun dev` | local Cloudflare dev via Alchemy | +| `cargo build --workspace` · `cargo test --workspace` | the Rust half | +| `bun run deploy:sandbox` | deploy a stage | -Existing top-level modules: `user/`, `team/`, `game/`, `pairing-code/`, `steam/`, `auth/`, `db/`. +## Two rules that are not style preferences -Modules that don't own their own table (like `steam/`) only need a single `index.ts` exposing reusable `fn()` functions — no `.sql.ts` file. +**Nothing closed may enter this repo.** Not source, not a dependency, not a +directory that "looked convenient". Before adding a top-level directory, know +which component it is and that the component is open. This has already been +caught once, in a commit that was never pushed. -## Pattern: `.sql.ts` (Drizzle Table) +**Versions are pinned once, centrally.** A Cargo member writes +`tokio.workspace = true` and never a version; a TS package uses the root +`catalog`. Two packages in one tree must not disagree about a dependency. -```ts -// src//.sql.ts -import { pgTable, text, boolean, jsonb, uniqueIndex, index, pgEnum } from 'drizzle-orm/pg-core'; +## Where the detail is -import { id, timestamps, ulid } from '../db/types.js'; +These load automatically when you work in the directory they describe — read +them there rather than duplicating them here. -// Enum (only if needed — co-located with its table) -export const SomeEnum = pgEnum('some_enum', ['a', 'b']); +- [`packages/core/CLAUDE.md`](packages/core/CLAUDE.md) — domain modules: the + `.sql.ts` / `index.ts` pair, `fn()`, the serialization boundary, ids, the + actor model, the error type, auth flow. +- [`apps/api/CLAUDE.md`](apps/api/CLAUDE.md) — route modules, registration, + `.meta()` vs `.openapi()`, error flow. +- [`docs/alchemy.md`](docs/alchemy.md) — infrastructure: stages, bindings, + secrets, service bindings, the CLI. -// FK imports — use the sql.ts files, never the index.ts (avoids circular deps) -import { UserTable } from '../user/user.sql.js'; +## Things worth knowing before you start -export const SomeTable = pgTable( - 'some_table', - { - ...id, // char(30) PK, prefix: som_ - ...timestamps, // time_created, time_updated, time_deleted (all utc) +**This tree is mid-rewrite.** The Rust components arrived recently, one commit +each, imported as trees rather than as history — so `git log` on them starts at +the import and their own past is not here. Docs for that half are thin and +being written. - // FK column — always use ulid() + .references() - userId: ulid('user_id') - .notNull() - .references(() => UserTable.id, { onDelete: 'cascade' }), +**The TypeScript half predates the Rust half**, so the guides above describe it +in much more depth. That is a gap in the writing, not a statement about which +half matters. - // Scalar columns - name: text('name').notNull(), - email: text('email'), // nullable = omit .notNull() - flag: boolean('flag').notNull().default(false), - metadata: jsonb('metadata').$type<{}>(), // JSON blob +**Rust components are guest-side**: they run inside a virtual machine, not on +the control plane. `nescope` composites, `nescapture` captures and encodes +frames from inside the workload's own process, `neswire` handles audio, and +`nesprotocol` is the wire format all three share. None of them talk to the API. - // Enum column - provider: SomeEnum('provider').notNull() - }, - (t) => [ - uniqueIndex('some_table_provider_unique').on(t.provider, t.providerAccountId), - index('some_table_sync_idx') - .on(t.userId) - .where(sql`${t.localValue} is distinct from ${t.remoteValue}`), - index('some_table_user_idx').on(t.userId) - ] -); -``` +## Conventions -### DB types (`src/db/types.ts`) - -| Helper | Output | -| ------------ | --------------------------------------------------------------- | -| `ulid(name)` | `char(30)` — for PKs and FKs | -| `id` | `{ id: ulid('id').primaryKey().notNull() }` — spread as `...id` | -| `utc(name)` | `timestamp with time zone` | -| `timestamps` | `{ timeCreated, timeUpdated (auto), timeDeleted }` | - -### Naming conventions - -- Table name: `snake_case` (e.g. `linked_account`, `team_member`) -- Column name: `snake_case` (e.g. `user_id`, `provider_account_id`, `time_created`) -- TypeScript field names: `camelCase` matching the column (drizzle maps them) -- Index names: `{table}_{column(s)}_unique` / `{table}_{column}_idx` - ---- - -## Pattern: `index.ts` (Domain Namespace) - -```ts -// src//index.ts -import { eq, and, isNull, sql } from 'drizzle-orm'; -import z from 'zod'; - -import { Database } from '../db/index.js'; -import { Examples } from '../examples.js'; -import { fn } from '../fn.js'; -import { SomeTable, SomeEnum } from './.sql.js'; - -export namespace SomeModule { - // ── Info schema ───────────────────────────────────────────────────── - // Single source of truth for the entity shape. - // Every field typed here; .meta() adds OpenAPI metadata. - // When a field changes here, TypeScript catches every usage. - export const Info = z - .object({ - id: z.string().meta({ - description: '…', - example: Examples.SomeModule.id, - }), - // For enum fields, use z.enum(SomeEnum.enumValues) to stay in sync: - provider: z.enum(SomeEnum.enumValues).meta({ … }), - // Nullable + optional for JSON-blob / optional fields: - metadata: z.record(z.string(), z.unknown()).nullable().optional().meta({ … }), - }) - .meta({ - ref: 'SomeModule', - description: '…', - example: Examples.SomeModule, - }); - - export type Info = z.infer; - - // ── create ─────────────────────────────────────────────────────────── - // Use Info.pick({…}) for the schema — keeps fields in sync with Info. - // Input is the parsed object. - // Use Database.use() for single-operation writes. - export const create = fn( - Info.pick({ id: true, name: true, email: true }), - async (input) => { - await Database.use(async (tx) => { - await tx.insert(SomeTable).values({ - id: input.id, - name: input.name, - email: input.email ?? null, - }); - }); - return input.id; - } - ); - - // ── Single-field lookups ───────────────────────────────────────────── - // Use Info.shape. for the schema. - // The callback receives the raw value, not { field: value }. - export const fromID = fn(Info.shape.id, async (id) => { - return Database.use(async (tx) => { - return tx - .select() - .from(SomeTable) - .where(and(eq(SomeTable.id, id), isNull(SomeTable.timeDeleted))) - .then((rows) => rows.at(0) ?? null); - }); - }); - - export const fromSlug = fn(Info.shape.slug, async (slug) => { - // … - }); - - // ── Multi-field lookups ────────────────────────────────────────────── - // Use Info.pick({ field1: true, field2: true }) - export const findByProvider = fn( - Info.pick({ provider: true, providerAccountId: true }), - async (input) => { - return Database.use(async (tx) => { - return tx - .select() - .from(SomeTable) - .where(and( - eq(SomeTable.provider, input.provider), - eq(SomeTable.providerAccountId, input.providerAccountId), - isNull(SomeTable.timeDeleted), - )) - .then((rows) => rows.at(0) ?? null); - }); - } - ); - - // ── List (no args) ─────────────────────────────────────────────────── - // Plain async function — no fn() wrapper since there's no input. - export async function list() { - return Database.use(async (tx) => { - return tx - .select() - .from(SomeTable) - .where(isNull(SomeTable.timeDeleted)) - .orderBy(SomeTable.timeCreated); - }); - } - - // ── List by FK ─────────────────────────────────────────────────────── - export const listByUser = fn(Info.shape.userId, async (userId) => { - return Database.use(async (tx) => { - return tx - .select() - .from(SomeTable) - .where(and(eq(SomeTable.userId, userId), isNull(SomeTable.timeDeleted))) - .orderBy(SomeTable.timeCreated); - }); - }); - - // ── Update ─────────────────────────────────────────────────────────── - // If the update uses Info fields + possibly extra fields: - export const updateSomething = fn( - Info.pick({ id: true, otherField: true }).extend({ - extraField: z.string(), - }), - async (input) => { - await Database.use(async (tx) => { - await tx - .update(SomeTable) - .set({ otherField: input.otherField }) - .where(eq(SomeTable.id, input.id)); - }); - } - ); - - // ── Soft-delete ────────────────────────────────────────────────────── - // Use sql\`now()\` for the timestamp (consistent with DB time). - export const remove = fn(Info.shape.id, async (id) => { - await Database.use(async (tx) => { - await tx - .update(SomeTable) - .set({ timeDeleted: sql`now()` }) - .where(eq(SomeTable.id, id)); - }); - }); - - // ── serialize ──────────────────────────────────────────────────────── - // Converts a DB row into the public API shape. Keeps serialization - // decoupled from the DB layer. - export function serialize(input: typeof SomeTable.$inferSelect): z.infer { - return { - id: input.id, - name: input.name, - // Cast enums since drizzle returns a string at runtime: - provider: input.provider as Info['provider'], - }; - } - - // ── listByUserWithGames ────────────────────────────────────────────── - // JOIN query with serialization inside the fn() boundary. - // The .then() chain maps raw rows to the public shape immediately. - export const listByUserWithGames = fn(Info.shape.userId, async (userId) => { - return Database.use(async (tx) => { - return tx - .select({ - library: UserLibraryTable, - game: GameTable - }) - .from(UserLibraryTable) - .leftJoin(GameTable, eq(UserLibraryTable.gameId, GameTable.id)) - .where(and(eq(UserLibraryTable.userId, userId), isNull(UserLibraryTable.timeDeleted))) - .orderBy(UserLibraryTable.timeCreated) - .then((rows) => - rows - .filter((row) => row.game !== null) - .map((row) => ({ - id: row.library.id, - game: Game.serialize(row.game!), - playtime2w: row.library.playtime2w, - playtimeForever: row.library.playtimeForever, - lastPlayed: row.library.lastPlayed?.toISOString() ?? null - })) - ); - }); - }); -} -``` - -### Key rules for `fn()` usage - -| Case | Schema | Callback receives | -| -------------------- | ------------------------------------------ | ------------------------------------- | -| Single field | `Info.shape.field` | Raw value (`string`, `boolean`, etc.) | -| Multiple fields | `Info.pick({a:true, b:true})` | `{ a, b }` object | -| Full entity | `Info` | Full `Info` object | -| Info fields + extras | `Info.pick({…}).extend({extra: z.type()})` | `{ …fields, extra }` | -| No input | Regular `async function` | n/a | - -### What `fn()` does - -```ts -fn(schema, callback); -// → (input) => { schema.parse(input); return callback(parsed); } -// The returned function also has a .schema property for OpenAPI introspection. -``` - ---- - -## Serialization Boundary Rule - -**All data transformation — including JOIN deserialization, field mapping, date stringification, and null filtering — happens INSIDE the `fn()` boundary, right after the DB query in a `.then()` chain. The API route is a dumb pass-through.** - -### Why this matters - -| Approach | Queries | Boundary clarity | -| ----------------------------------------------------- | ------------ | -------------------------------------- | -| **Bad**: Raw rows from core, map/filter in route | N+1 (or raw) | Leaky — route knows DB schema | -| **Bad**: JOIN in core, but serialize in route | 1 | Still leaky — route owns shaping logic | -| **Good**: JOIN + serialize in `.then()` inside `fn()` | 1 | Clean — core returns JSON-safe objects | - -### The pattern - -```ts -// GOOD: serialization inside fn() -export const listByUserWithGames = fn(Info.shape.userId, async (userId) => { - return Database.use(async (tx) => { - return tx - .select({ library: UserLibraryTable, game: GameTable }) - .from(UserLibraryTable) - .leftJoin(GameTable, eq(UserLibraryTable.gameId, GameTable.id)) - .where(...) - .orderBy(...) - .then((rows) => - rows - .filter((row) => row.game !== null) - .map((row) => ({ - id: row.library.id, - game: Game.serialize(row.game!), - lastPlayed: row.library.lastPlayed?.toISOString() ?? null - })) - ); - }); -}); - -// GOOD: API route is a thin pass-through -async (c) => { - const data = await Library.listByUserWithGames(Actor.userID); - return c.json({ data }); -} -``` - -### Rules - -1. **Never let raw Drizzle `$inferSelect` rows escape the core module.** If a function returns joined data, it must be shaped before the return. -2. **Use `.then()` after the query for map/filter/serialize.** Keeps the async pipeline declarative and co-located with the SQL. -3. **Re-use sibling `serialize()` functions for joined tables.** e.g. `Game.serialize(row.game!)` when joining `GameTable`. -4. **API routes only do:** auth checks, input validation, calling the core fn, and `c.json({ data })`. No `.map()`, no `.filter()`, no field remapping. - ---- - -### Mutation Rule: Single-Query Reads via `.returning()` - -Never execute a separate `tx.select()` or trigger a lookup function immediately after an `insert` or `upsert` mutation to fetch the updated state of a row. - -Postgres natively supports the `RETURNING` clause. Always append `.returning()` directly to your mutation chains and destructure the resulting array (`const [row] = await tx...`). This ensures mutations remain atomic, avoids unnecessary connection pool overhead, and removes the latency penalty of running two sequential database operations. - ---- - -## Pattern: ID generation (`src/id.ts`) - -```ts -Identifier.ascending('user') // → "usr_" -Identifier.ascending('team') // → "tem_" -Identifier.ascending('linkedAccount') // → "lac_" - -// Prefixes are defined in Identifier.prefixes: -{ - user: 'usr', - linkedAccount: 'lac', - team: 'tem', - teamMember: 'mem', - verification: 'ver', -} -``` - -The IDs are 30-char strings: `{prefix}_{26 base62 chars}`. They are monotonically increasing (time-sortable) when using `ascending()`. - ---- - -## Pattern: Examples (`src/examples.ts`) - -```ts -export namespace Examples { - export const Id = (prefix: keyof typeof Identifier.prefixes) => - `${Identifier.prefixes[prefix]}_XXXXXXXXXXXXXXXXXXXXXXXXX`; - - export const User = { id: Id('user'), name: '…', email: '…', … }; - export const LinkedAccount = { id: Id('linkedAccount'), provider: 'steam', … }; - export const Team = { id: Id('team'), slug: 'my-team', … }; - export const Member = { id: Id('teamMember'), role: 'owner' as const, … }; - export const Fingerprint = { id: Id('userFingerprint'), fingerprint: '…', … }; - export const GameDownload = { id: Id('gameDownload'), hostId: 'hst_…', gameId: Id('game'), status: 'downloading', … }; - export const Library = { id: Id('userLibrary'), playtimeForever: 150000, … }; - export const Depot = { id: Id('gameDepot'), depotId: 730, … }; -} -``` - -Every entity in `Examples` must be added and imported by the `Info` schema's `.meta({ example: … })`. - ---- - -## Pattern: Environment (`src/env.ts`) - -```ts -export namespace Env { - export const Info = z.object({ - NODE_ENV: z.enum(['development', 'production', 'test']).default('development'), - FRONTEND_URL: z.string().optional(), - STEAM_API_KEY: z.string().optional(), - AUTH_ISSUER_URL: z.string().optional() // used by API auth middleware to verify tokens - }); - export type Info = z.infer; - export const env: Info = Info.parse(process.env); -} -``` - ---- - -## Pattern: OpenAuth Subjects (`src/auth/subjects.ts`) - -```ts -import { createSubjects } from '@nestri/auth/subject'; -import { z } from 'zod'; - -export const subjects = createSubjects({ - user: z.object({ - userID: z.string(), - linkedAccountID: z.string() - }) -}); -``` - -The JWT contains `{ type: 'user', properties: { userID, linkedAccountID } }`. -The API verifies via `client.verify(subjects, token)` from `@nestri/auth/client`. - ---- - -## Entity relationships - -``` -User ───1:N─── LinkedAccount ← auth methods (Steam, Epic, etc.) - │ - ├─── 1:N ─── Fingerprint ← SSH public keys - ├─── 1:N ─── Library ← owned games with playtime - │ - └─── N:M ─── Team ← via Member - └─ role: owner | admin | member - -Game ───1:N─── Download ← per-host game depot downloads -``` - -- **User**: Person record. Email is nullable (gaming accounts don't provide one). -- **LinkedAccount**: A gaming/OAuth identity. `(provider, providerAccountId)` is unique. -- **Team**: Organization for billing/collaboration. First team is auto-created as "personal" team. -- **TeamMember**: Joins User → Team with a role. `(teamId, userId)` is unique. - ---- - -## Database access - -```ts -// Auto-scoped transaction (creates one if outside a transaction): -await Database.use(async (tx) => { - await tx.insert(SomeTable).values({ … }); - const result = await tx.select().from(SomeTable).where(…); -}); - -// Explicit transaction: -await Database.transaction(async (tx) => { - // All operations in one atomic transaction -}); - -// Side-effect queued after transaction commits: -Database.effect(() => sendEmail(…)); -``` - ---- - -## Soft-delete convention - -Every table has `time_deleted` (nullable timestamp). Queries filter with `isNull(table.timeDeleted)`. Deletion sets `timeDeleted: sql\`now()\`` — never hard-deletes. - ---- - -## Dependency rule - -- `.sql.ts` files may import from other `.sql.ts` files (for FK references). -- `index.ts` files may import from other `index.ts` files and `.sql.ts` files. -- Never import an `index.ts` from within a `.sql.ts` — that creates circular deps. - ---- - -## Actor Model (`packages/core/src/actor.ts`) - -The Actor model identifies who/what is making a request. It uses `AsyncLocalStorage` (via `Context.create()`) so the actor is accessible anywhere in the call chain without passing it around. - -### Actor types - -```ts -type ActorInfo = - | { type: 'public'; properties: {} } - | { type: 'user'; properties: { userID: string; linkedAccountID: string } } - | { - type: 'member'; - properties: { userID: string; teamID: string; role: 'owner' | 'admin' | 'member' }; - } - | { type: 'system'; properties: { teamID: string } } - | { type: 'admin'; properties: {} }; -``` - -### API - -```ts -Actor.use(); // → ActorInfo (throws if no context set) -Actor.with(value, fn); // Run fn in the given actor context -Actor.assert(type); // Assert current actor type, returns narrowed type -Actor.type; // → 'public' | 'user' | 'member' | 'system' | 'admin' -Actor.userID; // → string (user/member only) -Actor.linkedAccountID; // → string (user only) -Actor.useTeam; // → string (member/system only — the teamID) -Actor.role; // → 'owner' | 'admin' | 'member' (member only) -Actor.isSignedIn; // → boolean (true if not public) -``` - -### When to pull from Actor vs pass as param - -Functions that create resources owned by the current user (e.g. `Team.create`) pull `userID` from the actor context rather than requiring it as a parameter. This avoids passing `ownerId`/`userId` through the entire call chain. - -Domain functions that need the actor's identity import `Actor` and call `Actor.userID` inside the `fn()` callback: - -```ts -export const create = fn(Info.pick({ id: true, name: true, slug: true }), async (input) => { - const ownerId = Actor.userID; // from AsyncLocalStorage - // ... -}); -``` - -### Actor.userID inside Database.transaction() - -When doing find-or-create logic that must scope to the authenticated user, pull `Actor.userID` **inside** the `Database.transaction()` callback. This keeps scoping co-located with the DB logic: - -```ts -export const link = fn( - z.object({ steamId: z.string(), profile: z.record(z.string(), z.unknown()).optional() }), - async (input) => { - return Database.transaction(async () => { - const existing = await LinkedAccount.findByProvider({ ... }); - if (existing) return existing.id; - const id = Identifier.ascending('linkedAccount'); - await LinkedAccount.create({ id, userId: Actor.userID, provider: 'steam', ... }); - return id; - }); - } -); -``` - -This ensures the API endpoint is scoped to the user only — no `userId` param is passed from the route handler. The `Actor.userID` is set by the auth middleware's `Actor.with()`, propagated via `AsyncLocalStorage`. - -Callers must wrap actor-dependent work in `Actor.with()` before calling these functions. The API middleware does this automatically for HTTP requests. The auth worker sets it up explicitly: - -```ts -await Actor.with({ type: 'user', properties: { userID, linkedAccountID } }, async () => { - await Team.createPersonal({ displayName: personaname }); -}); -``` - ---- - -## Auth Flow - -### Auth Worker (`apps/auth/src/index.ts`) - -A Cloudflare Worker using `@nestri/auth` (OpenAuth). Entry point is the `success` callback after OAuth: - -1. Steam returns `steamid` → worker fetches profile from Steam API -2. Inside `Database.transaction()`: looks up existing `LinkedAccount.findByProvider`; if found returns existing user, else creates `User` + `LinkedAccount` -3. Wraps post-login setup in `Actor.with()`, then checks `Member.listByUser(userID)` -4. If no memberships, calls `Team.createPersonal({ displayName })` -5. Issues JWT via `context.subject('user', { userID, linkedAccountID })` - -### Admin Auth via Shared Secret - -Server-to-server calls can authenticate as an `admin` actor by setting the `x-nestri-admin-token` header to the value of `ADMIN_SHARED_SECRET`. This bypasses JWT auth entirely and grants a system-level actor with no user scope — useful for operations like adding games to the DB, syncing data, or other admin tasks. - -Configure `ADMIN_SHARED_SECRET` in `.env`; defaults to the same value as `SSH_AUTH_KEY` in dev (`dev-ssh-auth-key-change-in-prod`). - -### Auth Middleware (`apps/api/app/middleware/auth.ts`) - -Hono middleware that runs on every API request: - -1. Checks `x-nestri-admin-token` header — if it matches `Env.get().ADMIN_SHARED_SECRET`, sets actor to `admin` and proceeds immediately -2. Otherwise, reads `Authorization: Bearer ` header -3. Verifies via `client.verify(subjects, token)` from `@nestri/auth/client` -4. If valid `user` subject: - - Checks `x-nestri-team` header for team-scoped access - - If team header present, verifies membership via `Member.findByTeamAndUser` - - Sets actor to `member` (with role) or `user` type -5. If no token/invalid: sets actor to `public` type -6. Exports `notPublic` guard middleware — throws `VisibleError('authentication', UNAUTHORIZED, …)` if actor is `public`. Caught by `onError` → 401 JSON response. The `admin` actor passes this guard (it's not `public`), so admin routes can use `.use(notPublic)` like any other protected route. - -### OpenAuth Subjects (`src/auth/subjects.ts`) - -```ts -export const subjects = createSubjects({ - user: z.object({ userID: z.string(), linkedAccountID: z.string() }) -}); -``` - -The JWT contains `{ type: 'user', properties: { userID, linkedAccountID } }`. -Env var `AUTH_ISSUER_URL` configures which issuer to trust for token verification. - ---- - -## Team Discriminator System - -When creating a personal team (`Team.createPersonal`), the slug is derived from the display name: - -```ts -// 1. Sanitize: lowercase, replace non-alphanumeric with dashes, trim edges, max 50 chars -const baseSlug = displayName.toLowerCase().replace(/[^a-z0-9]+/g, '-')... - -// 2. Check if slug exists (fromSlug) -// 3. If taken, append random 4-digit discriminator: "name-1234" -const slug = existing ? `${baseSlug}-${discriminator}` : baseSlug; -``` - -This mirrors Discord's discriminator pattern. The discriminator is part of the slug string — not a separate column. - ---- - -## `fn()` and Actor Context - -`fn()` itself is unchanged — it validates input against a zod schema and calls the callback. The actor context is available inside `fn()` callbacks because `Actor.with()` uses `AsyncLocalStorage`, which propagates automatically through `await` chains. - -Every `fn()` callback can import and use `Actor` directly. No special wiring needed. - ---- - -## API Error Pattern (`packages/core/src/error.ts`) - -Centralized error types used by both the domain layer and the API. - -### ErrorResponse - -Zod schema for OpenAPI error responses: - -```ts -import { z } from 'zod'; - -export const ErrorResponse = z - .object({ - type: z.enum([ - 'validation', - 'authentication', - 'forbidden', - 'not_found', - 'already_exists', - 'rate_limit', - 'internal' - ]), - code: z.string(), - message: z.string(), - param: z.string().optional(), - details: z.any().optional() - }) - .meta({ ref: 'ErrorResponse' }); -``` - -### ErrorCodes - -Structured error code constants: - -| Category | Codes | -| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| `Validation` | `MISSING_REQUIRED_FIELD`, `ALREADY_EXISTS`, `TEAM_ALREADY_EXISTS`, `INVALID_PARAMETER`, `INVALID_FORMAT`, `INVALID_STATE`, `IN_USE` | -| `Authentication` | `UNAUTHORIZED`, `INVALID_TOKEN`, `EXPIRED_TOKEN`, `INVALID_CREDENTIALS` | -| `Permission` | `FORBIDDEN`, `INSUFFICIENT_PERMISSIONS`, `ACCOUNT_RESTRICTED` | -| `NotFound` | `RESOURCE_NOT_FOUND` | -| `RateLimit` | `TOO_MANY_REQUESTS`, `QUOTA_EXCEEDED` | -| `Server` | `INTERNAL_ERROR`, `SERVICE_UNAVAILABLE`, `DEPENDENCY_FAILURE` | - -### VisibleError - -Throw this for any user-facing error. It carries structured data and converts cleanly to HTTP: - -```ts -throw new VisibleError( - 'not_found', - ErrorCodes.NotFound.RESOURCE_NOT_FOUND, - `User ${id} does not exist` -); -``` - -- `.statusCode()` — maps `type` → HTTP status (validation→400, authentication→401, forbidden→403, not_found→404, already_exists→409, rate_limit→429, internal→500) -- `.toResponse()` → `{ type, code, message, param?, details? }` - -The API's global `onError` handler catches `VisibleError` + `HTTPException` + unknown errors, logging each and returning the correct JSON shape. - ---- - -## API Route Pattern (`apps/api/app/routes/`) - -Every API domain is a TypeScript `namespace` with a `.route` property — a plain `new Hono()` instance with chained route definitions. - -### Route → Domain function flow - -``` -HTTP request ──► Route handler (thin) ──► Core domain fn() ──► DB - │ │ - │ validates input │ pulls Actor.userID - │ calls domain fn │ handles business logic - │ returns c.json({data}) │ inside Database.transaction() -``` - -Route handlers are **thin wrappers** — they validate input, call a core function, and return the result. All business logic lives in `packages/core/src//`. - -### Creating a route module - -```ts -// app/routes/.ts -import { z } from 'zod'; -import { Hono } from 'hono'; -import { describeRoute } from 'hono-openapi'; -import { Thing } from '@nestri/core/thing/index'; -import { Examples } from '@nestri/core/examples'; -import { ErrorCodes, VisibleError } from '@nestri/core/error'; -import { ErrorResponses, notPublic, Result, validator } from '../utils'; - -export namespace ThingApi { - export const route = new Hono() - .use(notPublic) - .get( - '/', - describeRoute({ - tags: ['Thing'], - summary: 'List things', - description: 'List all things', - responses: { - 200: { - content: { - 'application/json': { - schema: Result( - Thing.Info.array().meta({ - description: 'All things', - example: [Examples.Thing] - }) - ) - } - }, - description: 'All things' - }, - 400: ErrorResponses[400], - 404: ErrorResponses[404], - 429: ErrorResponses[429] - } - }), - async (c) => c.json({ data: await Thing.list() }) - ) - .get( - '/:id', - describeRoute({/* … */}), - validator( - 'param', - z.object({ - id: z.string().meta({ - description: 'ID of the thing', - example: Examples.Thing.id - }) - }) - ), - async (c) => { - const thing = await Thing.fromID(c.req.valid('param').id); - if (!thing) { - throw new VisibleError( - 'not_found', - ErrorCodes.NotFound.RESOURCE_NOT_FOUND, - `Thing ${id} not found` - ); - } - return c.json({ data: thing }); - } - ); -} -``` - -### Grouped routes: `/(group)/(sub-route)` - -For domains with multiple sub-routes (e.g. Steam with `/link`, `/sync`, `/unlink`), group them under one route file. The namespace name is `XxxApi` (e.g. `SteamApi`), the route path is `/(group)`: - -```ts -// app/routes/steam.ts -import { z } from "zod"; -import { Hono } from "hono"; -import { describeRoute } from "hono-openapi"; -import { Steam } from "@nestri/core/steam/index"; -import { ErrorResponses, notPublic, Result, validator } from "../utils"; - -export namespace SteamApi { - export const route = new Hono() - .use(notPublic) - .post("/link", // → POST /steam/link - describeRoute({ tags: ["Steam"], summary: "Link a Steam account", ... }), - validator("json", z.object({ steamId: z.string() })), - async (c) => { - const { steamId } = c.req.valid("json"); - const result = await Steam.link({ steamId }); // ← calls core fn - return c.json({ data: { linkedAccountId: result, steamId } }); - }, - ) - .post("/sync", // → POST /steam/sync - // ... - ); -} -``` - -Registered in the app entry as `/steam`: - -```ts -// app/index.ts -import { SteamApi } from "./routes/steam.js"; - -const routes = app - .route("/", IndexApi.route) - .route("/users", UserApi.route) - .route("/steam", SteamApi.route) // mounts all /steam/* routes - .onError(…); -``` - -This keeps the route path and the namespace name aligned — the Hono instance at `SteamApi.route` is mounted at `/steam`. - -### Key conventions - -| Element | Pattern | -| ---------------- | --------------------------------------------------------------------------------- | -| Structure | `export namespace XxxApi { export const route = new Hono() … }` | -| Group route | `POST "/link"` at `XxxApi` → mounted at `/xxx` → `POST /xxx/link` | -| Auth guard | `.use(notPublic)` at the namespace level (or per-route) | -| Route is thin | validates input → calls core fn → returns `c.json({ data: … })` | -| Core fn | reusable `fn()` in `packages/core/src//` owns all logic | -| OpenAPI | `describeRoute({ tags, summary, description, responses })` wraps each handler | -| Response schema | `Result(Schema)` → `resolver(z.object({ data: schema }))` | -| Error responses | `ErrorResponses[statusCode]` for 400, 401, 403, 404, 409, 429, 500 | -| Param validation | `validator("param", z.object({…}))` — uses custom wrapper that formats Zod errors | -| Body validation | `validator("json", z.object({…}))` — same wrapper for request body | -| Not found | `throw new VisibleError("not_found", ErrorCodes.NotFound.RESOURCE_NOT_FOUND, …)` | -| Metadata | Use `.meta()` (NOT `.openapi()`) — Zod v4 native + `zod-openapi` v6 | - -### Registering a route in the app - -```ts -// app/index.ts -import { SteamApi } from "./routes/steam.js"; -import { ThingApi } from "./routes/thing.js"; - -const routes = app - .route("/", IndexApi.route) - .route("/users", UserApi.route) - .route("/steam", SteamApi.route) // mount group at /steam - .route("/things", ThingApi.route) // mount group at /things - .onError(…); -``` - -The first argument to `.route()` is the URL prefix. All sub-routes defined on that Hono instance are relative to this prefix. - ---- - -## API Utils (`apps/api/app/utils/`) - -| File | Export | Purpose | -| -------------- | -------------------- | ---------------------------------------------------------------------------------------------- | -| `index.ts` | — | Barrel re-export of all utils | -| `auth.ts` | `auth`, `notPublic` | Re-exports from `middleware/auth` | -| `error.ts` | `ErrorResponses` | `{ 400, 401, 403, 404, 409, 429, 500 }` → OpenAPI response objects | -| `result.ts` | `Result` | `resolver(z.object({ data: T }))` — standard `{ data: … }` response shape | -| `validator.ts` | `validator` | Wraps `hono-openapi/zod`'s validator with standardized Zod error formatting (400 + error code) | -| `hook.ts` | `Hook`, `zValidator` | Type declarations (re-exported from `@hono/zod-validator`) | - ---- - -## Main API Entry (`apps/api/app/index.ts`) - -The entry point wires everything together. Key structure: - -```ts -import 'zod-openapi'; // augment Zod v4 with OpenAPI metadata types -import { Hono } from 'hono'; -import { logger } from 'hono/logger'; -import { cors } from 'hono/cors'; -import { showRoutes } from 'hono/dev'; -import { openAPISpecs } from 'hono-openapi'; -import { HTTPException } from 'hono/http-exception'; - -export const app = new Hono(); - -// Global middleware (order matters) -app - .use(logger()) - .use(async (c, next) => { - c.header('Cache-Control', 'no-store'); - return next(); - }) - .use(cors({ origin: Env.env.FRONTEND_URL || 'http://localhost:5173', credentials: true })) - .use(auth); - -// Routes + error handler -const routes = app - .route('/', IndexApi.route) - .route('/things', ThingApi.route) - .onError((error, c) => { - if (error instanceof VisibleError) { - return c.json(error.toResponse(), error.statusCode()); - } - if (error instanceof HTTPException) { - return c.json( - { - type: 'validation', - code: ErrorCodes.Validation.INVALID_PARAMETER, - message: 'Invalid request' - }, - error.status - ); - } - return c.json( - { - type: 'internal', - code: ErrorCodes.Server.INTERNAL_ERROR, - message: 'Internal server error' - }, - 500 - ); - }); - -// OpenAPI spec at /doc -app.get( - '/doc', - openAPISpecs(routes, { documentation: { info: { title: 'API', version: '0.0.1' } } }) -); - -showRoutes(app); - -export default { port: process.env.PORT ?? 3000, fetch: app.fetch }; -``` - -### Dev / production - -``` -bun --watch app/index.ts # dev with hot reload -bun app/index.ts # production -``` - -No Vite needed — Bun runs TypeScript natively. - ---- - -## Important: `.meta()` vs `.openapi()` - -| Library | Method | Notes | -| -------------------------- | ------------ | ------------------------------------------------------------------------------------ | -| `zod-openapi` v4 (old) | `.openapi()` | Required `import "zod-openapi/extend"` | -| `zod-openapi` v6 (current) | `.meta()` | Native Zod v4 method; no import needed — auto-augments via `declare module 'zod/v4'` | - -- **Domain schemas** (`@nestri/core/*/index.ts`) use `.meta()` for descriptions/examples. -- **API route schemas** (`apps/api/app/routes/*.ts`) use `.meta()` for OpenAPI response/docs metadata. -- **Never** use `.openapi()` — it doesn't exist in `zod-openapi` v6. - ---- - -## Error flow summary - -``` -Route handler - │ - ├─ throws VisibleError ──► onError → c.json(error.toResponse(), error.statusCode()) - │ - ├─ throws HTTPException ─► onError → c.json({ type: "validation", … }, error.status) - │ - ├─ throws raw Error ─────► onError → c.json({ type: "internal", … }, 500) - │ (includes VisibleError for Actor model / context issues) - │ - └─ returns normally ─────► c.json({ data: … }) -``` - ---- - -# Alchemy (IaC) - -This project uses [Alchemy](https://alchemy.run) (v0.93.12) for infrastructure-as-code — the equivalent of SST, but targeting Cloudflare Workers instead of AWS Lambda. - -## Project structure - -``` -web/ - alchemy.run.ts # Entry point — creates scope, imports infra - infra/ - stage.ts # Stage detection (Scope.getCurrentScope().stage) - secret.ts # Encrypted secrets via alchemy.secret() - auth.ts # Auth Worker resource - api.ts # API Worker resource -``` - -## `alchemy.run.ts` — entry point - -```ts -import alchemy from 'alchemy'; - -const app = await alchemy('nestri', { - password: process.env.ALCHEMY_PASSWORD // required for secrets -}); - -// Import infra modules in dependency order (SST-style) -await import('./infra/stage.ts'); -await import('./infra/secret.ts'); -await import('./infra/auth.ts'); -await import('./infra/api.ts'); - -await app.finalize(); -``` - -Key rules: - -- `alchemy(appName, opts)` creates a **scope** — resources register into this scope automatically -- `app.finalize()` must be called at the end to persist state -- Import order matters — resources that depend on others must be imported after -- `--dev` flag runs locally via Miniflare; omit it to deploy to Cloudflare - -## Infra resources - -Each resource is imported from `alchemy/cloudflare` and called with an ID + props: - -```ts -import { Worker, KVNamespace, D1Database } from 'alchemy/cloudflare'; - -export const kv = await KVNamespace('my-kv'); -export const db = await D1Database('my-db'); - -export const worker = await Worker('my-worker', { - entrypoint: 'apps/some-app/src/index.ts', - compatibility: 'node', // enables nodejs_compat flag - url: true, // assign workers.dev URL - bindings: { - KV: kv, // resource binding → KVNamespace at runtime - DB: db, // → D1Database - PLAIN_VAR: 'hello' // → plain_text binding - } -}); -``` - -### Supported resources (subset) - -| Resource | Import | Purpose | -| ------------- | -------------------- | ----------------------------------------------- | -| `Worker` | `alchemy/cloudflare` | Cloudflare Worker (entrypoint or inline script) | -| `KVNamespace` | `alchemy/cloudflare` | KV storage | -| `D1Database` | `alchemy/cloudflare` | D1 SQL database | -| `R2Bucket` | `alchemy/cloudflare` | R2 object storage | -| `Queue` | `alchemy/cloudflare` | Queue/pub-sub | - -### Compatibility flag - -Always add `compatibility: 'node'` to Workers that use Node.js built-ins (`node:async_hooks`, `crypto`, `node:stream`, etc.): - -```ts -Worker('api', { - entrypoint: 'apps/api/app/index.ts', - compatibility: 'node' // enables nodejs_compat -}); -``` - -## Stage detection - -```ts -// infra/stage.ts -import { Scope } from 'alchemy'; -const scope = Scope.getCurrentScope(); -export const stage = scope?.stage ?? 'dev'; -export const isPermanent = ['production', 'dev'].includes(stage); -``` - -Use stage for conditional infrastructure: - -```ts -const api = await Worker('api', { - ...(isPermanent && { - observability: { enabled: true }, - logpush: true - }) -}); -``` - -Pass `--stage` flag at runtime: `bun alchemy.run.ts --stage production` - -## Secrets and environment variables - -Three levels of env management, from most-secure to least: - -### 1. `alchemy.secret.env.X` (preferred) - -```ts -// infra/secret.ts -import alchemy from 'alchemy'; - -export const secret = { - steamApiKey: alchemy.secret.env.STEAM_API_KEY // reads process.env at deploy time - // Equivalent to: - // steamApiKey: alchemy.secret(process.env.STEAM_API_KEY), -}; -``` - -- Reads from `process.env` at deploy time -- Throws a descriptive error if the env var is missing -- Encrypted in Alchemy state files (`.alchemy/`) -- Deployed as `secret_text` binding (hidden from Cloudflare API) - -### 2. `alchemy.env()` (non-secret config) - -```ts -export const frontendUrl = alchemy.env('FRONTEND_URL', 'http://localhost:5173'); -``` - -- Optional default value -- Plain text — not encrypted -- Deployed as `plain_text` binding - -### 3. Plain strings in `bindings` (inline) - -```ts -bindings: { - MY_VAR: 'hello'; -} -``` - -- Hard-coded, visible in state files -- Deployed as `plain_text` binding - -### How bindings map to runtime types - -| Alchemy binding type | Deployed as | Runtime type | -| -------------------- | -------------- | -------------------------- | -| `Worker` | `service` | `Service` (has `.fetch()`) | -| `KVNamespace` | `kv_namespace` | `KVNamespace` | -| `D1Database` | `d1` | `D1Database` | -| `alchemy.secret()` | `secret_text` | `string` | -| plain `string` | `plain_text` | `string` | -| `Json(...)` | `json` | `typeof json` | - -## Service bindings (Worker → Worker) - -Pass one Worker as a binding to another: - -```ts -// infra/auth.ts -export const auth = await Worker('auth', { - entrypoint: 'apps/auth/src/index.ts', - compatibility: 'node', - bindings: { ... }, -}); - -// infra/api.ts -import { auth } from './auth.ts'; -export const api = await Worker('api', { - entrypoint: 'apps/api/app/index.ts', - bindings: { AUTH: auth }, -}); -``` - -At runtime, `env.AUTH` is a `Service` — call it directly: - -```ts -const response = await env.AUTH.fetch(request); -``` - -### OpenAuth client + service binding - -The `@openauthjs/openauth/client` only accepts a URL string for `issuer`, so use a custom `fetch` to route through the service binding: - -```ts -function getClient(env: Record) { - return createClient({ - issuer: 'https://auth.internal', // dummy — used for path construction - clientID: 'api', - fetch: (input, init) => { - const url = new URL(typeof input === 'string' ? input : input.url); - const request = new Request(url.pathname + url.search, init); - return (env.AUTH as { fetch: typeof fetch }).fetch(request); - } - }); -} -``` - -## Env propagation to Workers - -CF Workers receive env vars as the second argument to the `fetch` handler (`env`), NOT via `process.env`. Bridge the gap with a lazy + overridable schema: - -```ts -// packages/core/src/env.ts -import { memo } from '../utils/memo.ts'; - -let _overrides: Record = {}; - -export namespace Env { - export const Info = z.object({ - FRONTEND_URL: z.string().optional(), - STEAM_API_KEY: z.string().optional(), - AUTH_ISSUER_URL: z.string().optional() - }); - export type Info = z.infer; - - const _get = memo(() => Info.parse({ ...process.env, ..._overrides })); - - export function get(): Info { - return _get(); - } - - export function init(bindings: Record) { - _overrides = bindings; - _get.reset(); - } -} -``` - -Wire in the Hono entrypoint: - -```ts -export default { - fetch(request, env, ctx) { - Env.init(env); // merge CF bindings into Env - return app.fetch(request, env, ctx); - } -}; -``` - -Now any module that imports `Env.get()` gets the correct values — on Bun dev `process.env` provides them, on CF Workers the bindings override. - -## CLI usage - -```sh -# Local dev (Miniflare) -bun alchemy.run.ts --dev - -# Deploy to Cloudflare -bun alchemy.run.ts --stage production - -# Destroy all resources -bun alchemy.run.ts --destroy - -# With custom stage -bun alchemy.run.ts --stage wanjohiryan - -# Password (for encrypting secrets) -export ALCHEMY_PASSWORD="some-passphrase" -``` - -When deploying, set `CLOUDFLARE_API_TOKEN` or configure `alchemy login`. - -## Common patterns - -### Conditional infra per-stage - -```ts -Worker('api', { - ...(isPermanent && { logpush: true }), - ...(stage === 'production' && { scaling: { min: 3, max: 10 } }) -}); -``` - -### Across-app resource references - -Alchemy uses top-level await in infra files — resources resolve at import time within the active scope. The scope propagates via `AsyncLocalStorage`, so any `await import()` after `alchemy(appName)` picks it up. - -### .alchemy/ directory - -Created automatically — contains Miniflare state, build output, and encrypted state files. Add to `.gitignore`. - -```gitignore -.alchemy/ -``` - ---- - -### Index Rule: Null-Safe Exclusions (`IS DISTINCT FROM`) - -When writing indices to track data drift, synchronization deltas, or pending background worker states where values might be nullable, **always build a partial index utilizing Postgres-native `IS DISTINCT FROM`**. - -Standard inequality operators (`!=` or `<>`) evaluate to `NULL` if either column is `NULL`, causing them to bypass standard `WHERE` index filters. Using `is distinct from` allows Postgres to treat `NULL` as a real value for state comparison: - -- Excludes perfectly synchronized records completely from the index footprint. -- Optimizes heavy background worker poll queries directly into small, lightning-fast index scans. - -2. Add to the Pattern: index.ts (Domain Namespace) section - -Replace the existing create block and add the upsert block inside SomeModule: - -```ts -// ── create ─────────────────────────────────────────────────────────── -// Use Info.pick({…}) for the schema — keeps fields in sync with Info. -// Always use .returning() to get the updated row context in one database trip. -export const create = fn(Info.pick({ id: true, name: true, email: true }), async (input) => { - return Database.use(async (tx) => { - const [row] = await tx - .insert(SomeTable) - .values({ - id: input.id, - name: input.name, - email: input.email ?? null - }) - .returning(); - return row; - }); -}); - -// ── upsert ─────────────────────────────────────────────────────────── -// Simple copies use the input values directly. For coalesce-style set -// expressions, reference the excluded pseudo-table with unqualified -// identifiers: sql`excluded.${sql.identifier(SomeTable.name.name)}` — -// interpolating a column object (or its .name string) is invalid. -export const upsert = fn(Info.pick({ id: true, name: true }), async (input) => { - return Database.use(async (tx) => { - const [row] = await tx - .insert(SomeTable) - .values({ id: input.id, name: input.name }) - .onConflictDoUpdate({ - target: SomeTable.id, - set: { name: input.name } - }) - .returning(); - return row; - }); -}); -``` +Conventional commits. Explain *why* in the body — the diff already shows what. +Comments earn their place by saying something the code cannot; a comment +restating the line below it is noise. diff --git a/README.md b/README.md index aed49609..aec40ae6 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,83 @@

-

Game streaming platform — play your games from any device via QUIC low-latency streams.

+

Run your games on a GPU you don't own — or one you do. Nestri puts an interactive workload in a hardware-accelerated virtual machine and streams it to you over QUIC, at a latency that lets you play rather than watch.

-- **Streaming core** — QUIC-based relay, game machines, pairing codes -- **Games** — Steam-linked catalog with per-machine depot downloads -- **Auth** — Steam / SSH login via a self-hosted OpenAuth issuer -- **Infra** — Cloudflare Workers + Postgres, deployed with Alchemy +> [!NOTE] +> **This repository is mid-rewrite, and the documentation is behind the code.** +> The guest-side components arrived recently and their docs are thin. Nothing +> here is stable yet: expect directories to move and interfaces to change. +> Proper documentation is on the way — issues and questions are welcome in the +> meantime, and are genuinely useful for deciding what to write first. + +## What is here + +Two halves that meet over the network and share very little else. + +### The control plane — TypeScript, on Cloudflare Workers + +| | | +|---|---| +| [`apps/api`](apps/api) | The public REST API. Identity, teams, machines, games, pairing. | +| [`apps/auth`](apps/auth) | A self-hosted OpenAuth issuer — Steam and SSH-key login. | +| [`packages/core`](packages/core) | The domain: every table, every operation, no HTTP. | +| [`packages/auth`](packages/auth) | Shared auth types and subjects. | + +Postgres for state, [Alchemy](https://alchemy.run) for infrastructure. See +[`docs/alchemy.md`](docs/alchemy.md). + +### The guest — Rust, inside the box + +These run *inside* a virtual machine, beside the game. None of them talk to the +control plane. + +| | | +|---|---| +| [`apps/nescope`](apps/nescope) | A headless Wayland compositor for one fullscreen client. A lighter answer to the same problem gamescope solves. | +| [`apps/nescapture`](apps/nescapture) | A Vulkan implicit layer. It captures frames from inside the workload's own process and encodes them on the GPU that drew them — no copy out to the CPU and back. | +| [`apps/neswire`](apps/neswire) | Audio capture and transport. | +| [`crates/nesprotocol`](crates/nesprotocol) | The wire types all three share, so no two ends can drift apart silently. | + +The hypervisor these run under is [`nesbox`](https://github.com/nestrilabs/nesbox), +a separate repository: a micro-VM with a real GPU in it, using virtio-gpu native +context rather than passthrough, so one card can host several boxes at once. + +## Why a virtual machine + +A container shares the host kernel, which makes strong isolation hard and a GPU +harder. A micro-VM boots in about as long, isolates properly, and — with native +context — gets close to bare-metal graphics. That choice is what makes "many +sandboxes, one GPU" possible instead of one tenant per card. + +## Getting started + +```sh +bun install +bun dev # control plane, local Cloudflare runtime + +cargo build --workspace # guest components +cargo test --workspace +``` + +The Rust components expect a Linux host with a Wayland-capable GPU stack. They +are not much use on their own yet — they are pieces of a box, and the thing that +assembles a box is not open yet. + +## Status + +Working: the API, auth, the domain model, and the guest components listed above. + +Not here yet: the box lifecycle, storage, the edge, and the client. Some of that +will open as it is written; some is deliberately closed. What decides which is +whether it handles your data — that half is open on principle — or decides our +capacity, which is the part we sell. + +## Contributing + +Early, and the ground moves. The most useful contribution right now is telling +us where the documentation failed you. Conventional commits; explain *why* in +the body. + +## Licence + +[Apache 2.0](LICENSE). diff --git a/apps/api/CLAUDE.md b/apps/api/CLAUDE.md new file mode 100644 index 00000000..d948074f --- /dev/null +++ b/apps/api/CLAUDE.md @@ -0,0 +1,284 @@ +# `apps/api` — routes, errors and the entry point + +## API Route Pattern (`apps/api/app/routes/`) + +Every API domain is a TypeScript `namespace` with a `.route` property — a plain `new Hono()` instance with chained route definitions. + +### Route → Domain function flow + +``` +HTTP request ──► Route handler (thin) ──► Core domain fn() ──► DB + │ │ + │ validates input │ pulls Actor.userID + │ calls domain fn │ handles business logic + │ returns c.json({data}) │ inside Database.transaction() +``` + +Route handlers are **thin wrappers** — they validate input, call a core function, and return the result. All business logic lives in `packages/core/src//`. + +### Creating a route module + +```ts +// app/routes/.ts +import { z } from 'zod'; +import { Hono } from 'hono'; +import { describeRoute } from 'hono-openapi'; +import { Thing } from '@nestri/core/thing/index'; +import { Examples } from '@nestri/core/examples'; +import { ErrorCodes, VisibleError } from '@nestri/core/error'; +import { ErrorResponses, notPublic, Result, validator } from '../utils'; + +export namespace ThingApi { + export const route = new Hono() + .use(notPublic) + .get( + '/', + describeRoute({ + tags: ['Thing'], + summary: 'List things', + description: 'List all things', + responses: { + 200: { + content: { + 'application/json': { + schema: Result( + Thing.Info.array().meta({ + description: 'All things', + example: [Examples.Thing] + }) + ) + } + }, + description: 'All things' + }, + 400: ErrorResponses[400], + 404: ErrorResponses[404], + 429: ErrorResponses[429] + } + }), + async (c) => c.json({ data: await Thing.list() }) + ) + .get( + '/:id', + describeRoute({/* … */}), + validator( + 'param', + z.object({ + id: z.string().meta({ + description: 'ID of the thing', + example: Examples.Thing.id + }) + }) + ), + async (c) => { + const thing = await Thing.fromID(c.req.valid('param').id); + if (!thing) { + throw new VisibleError( + 'not_found', + ErrorCodes.NotFound.RESOURCE_NOT_FOUND, + `Thing ${id} not found` + ); + } + return c.json({ data: thing }); + } + ); +} +``` + +### Grouped routes: `/(group)/(sub-route)` + +For domains with multiple sub-routes (e.g. Steam with `/link`, `/sync`, `/unlink`), group them under one route file. The namespace name is `XxxApi` (e.g. `SteamApi`), the route path is `/(group)`: + +```ts +// app/routes/steam.ts +import { z } from "zod"; +import { Hono } from "hono"; +import { describeRoute } from "hono-openapi"; +import { Steam } from "@nestri/core/steam/index"; +import { ErrorResponses, notPublic, Result, validator } from "../utils"; + +export namespace SteamApi { + export const route = new Hono() + .use(notPublic) + .post("/link", // → POST /steam/link + describeRoute({ tags: ["Steam"], summary: "Link a Steam account", ... }), + validator("json", z.object({ steamId: z.string() })), + async (c) => { + const { steamId } = c.req.valid("json"); + const result = await Steam.link({ steamId }); // ← calls core fn + return c.json({ data: { linkedAccountId: result, steamId } }); + }, + ) + .post("/sync", // → POST /steam/sync + // ... + ); +} +``` + +Registered in the app entry as `/steam`: + +```ts +// app/index.ts +import { SteamApi } from "./routes/steam.js"; + +const routes = app + .route("/", IndexApi.route) + .route("/users", UserApi.route) + .route("/steam", SteamApi.route) // mounts all /steam/* routes + .onError(…); +``` + +This keeps the route path and the namespace name aligned — the Hono instance at `SteamApi.route` is mounted at `/steam`. + +### Key conventions + +| Element | Pattern | +| ---------------- | --------------------------------------------------------------------------------- | +| Structure | `export namespace XxxApi { export const route = new Hono() … }` | +| Group route | `POST "/link"` at `XxxApi` → mounted at `/xxx` → `POST /xxx/link` | +| Auth guard | `.use(notPublic)` at the namespace level (or per-route) | +| Route is thin | validates input → calls core fn → returns `c.json({ data: … })` | +| Core fn | reusable `fn()` in `packages/core/src//` owns all logic | +| OpenAPI | `describeRoute({ tags, summary, description, responses })` wraps each handler | +| Response schema | `Result(Schema)` → `resolver(z.object({ data: schema }))` | +| Error responses | `ErrorResponses[statusCode]` for 400, 401, 403, 404, 409, 429, 500 | +| Param validation | `validator("param", z.object({…}))` — uses custom wrapper that formats Zod errors | +| Body validation | `validator("json", z.object({…}))` — same wrapper for request body | +| Not found | `throw new VisibleError("not_found", ErrorCodes.NotFound.RESOURCE_NOT_FOUND, …)` | +| Metadata | Use `.meta()` (NOT `.openapi()`) — Zod v4 native + `zod-openapi` v6 | + +### Registering a route in the app + +```ts +// app/index.ts +import { SteamApi } from "./routes/steam.js"; +import { ThingApi } from "./routes/thing.js"; + +const routes = app + .route("/", IndexApi.route) + .route("/users", UserApi.route) + .route("/steam", SteamApi.route) // mount group at /steam + .route("/things", ThingApi.route) // mount group at /things + .onError(…); +``` + +The first argument to `.route()` is the URL prefix. All sub-routes defined on that Hono instance are relative to this prefix. + +--- + +## API Utils (`apps/api/app/utils/`) + +| File | Export | Purpose | +| -------------- | -------------------- | ---------------------------------------------------------------------------------------------- | +| `index.ts` | — | Barrel re-export of all utils | +| `auth.ts` | `auth`, `notPublic` | Re-exports from `middleware/auth` | +| `error.ts` | `ErrorResponses` | `{ 400, 401, 403, 404, 409, 429, 500 }` → OpenAPI response objects | +| `result.ts` | `Result` | `resolver(z.object({ data: T }))` — standard `{ data: … }` response shape | +| `validator.ts` | `validator` | Wraps `hono-openapi/zod`'s validator with standardized Zod error formatting (400 + error code) | +| `hook.ts` | `Hook`, `zValidator` | Type declarations (re-exported from `@hono/zod-validator`) | + +--- + +## Main API Entry (`apps/api/app/index.ts`) + +The entry point wires everything together. Key structure: + +```ts +import 'zod-openapi'; // augment Zod v4 with OpenAPI metadata types +import { Hono } from 'hono'; +import { logger } from 'hono/logger'; +import { cors } from 'hono/cors'; +import { showRoutes } from 'hono/dev'; +import { openAPISpecs } from 'hono-openapi'; +import { HTTPException } from 'hono/http-exception'; + +export const app = new Hono(); + +// Global middleware (order matters) +app + .use(logger()) + .use(async (c, next) => { + c.header('Cache-Control', 'no-store'); + return next(); + }) + .use(cors({ origin: Env.env.FRONTEND_URL || 'http://localhost:5173', credentials: true })) + .use(auth); + +// Routes + error handler +const routes = app + .route('/', IndexApi.route) + .route('/things', ThingApi.route) + .onError((error, c) => { + if (error instanceof VisibleError) { + return c.json(error.toResponse(), error.statusCode()); + } + if (error instanceof HTTPException) { + return c.json( + { + type: 'validation', + code: ErrorCodes.Validation.INVALID_PARAMETER, + message: 'Invalid request' + }, + error.status + ); + } + return c.json( + { + type: 'internal', + code: ErrorCodes.Server.INTERNAL_ERROR, + message: 'Internal server error' + }, + 500 + ); + }); + +// OpenAPI spec at /doc +app.get( + '/doc', + openAPISpecs(routes, { documentation: { info: { title: 'API', version: '0.0.1' } } }) +); + +showRoutes(app); + +export default { port: process.env.PORT ?? 3000, fetch: app.fetch }; +``` + +### Dev / production + +``` +bun --watch app/index.ts # dev with hot reload +bun app/index.ts # production +``` + +No Vite needed — Bun runs TypeScript natively. + +--- + +## Important: `.meta()` vs `.openapi()` + +| Library | Method | Notes | +| -------------------------- | ------------ | ------------------------------------------------------------------------------------ | +| `zod-openapi` v4 (old) | `.openapi()` | Required `import "zod-openapi/extend"` | +| `zod-openapi` v6 (current) | `.meta()` | Native Zod v4 method; no import needed — auto-augments via `declare module 'zod/v4'` | + +- **Domain schemas** (`@nestri/core/*/index.ts`) use `.meta()` for descriptions/examples. +- **API route schemas** (`apps/api/app/routes/*.ts`) use `.meta()` for OpenAPI response/docs metadata. +- **Never** use `.openapi()` — it doesn't exist in `zod-openapi` v6. + +--- + +## Error flow summary + +``` +Route handler + │ + ├─ throws VisibleError ──► onError → c.json(error.toResponse(), error.statusCode()) + │ + ├─ throws HTTPException ─► onError → c.json({ type: "validation", … }, error.status) + │ + ├─ throws raw Error ─────► onError → c.json({ type: "internal", … }, 500) + │ (includes VisibleError for Actor model / context issues) + │ + └─ returns normally ─────► c.json({ data: … }) +``` + diff --git a/docs/alchemy.md b/docs/alchemy.md new file mode 100644 index 00000000..e1712fe7 --- /dev/null +++ b/docs/alchemy.md @@ -0,0 +1,345 @@ +# Alchemy — infrastructure as code + +This project uses [Alchemy](https://alchemy.run) (v0.93.12) for infrastructure-as-code — the equivalent of SST, but targeting Cloudflare Workers instead of AWS Lambda. + +## Project structure + +``` +web/ + alchemy.run.ts # Entry point — creates scope, imports infra + infra/ + stage.ts # Stage detection (Scope.getCurrentScope().stage) + secret.ts # Encrypted secrets via alchemy.secret() + auth.ts # Auth Worker resource + api.ts # API Worker resource +``` + +## `alchemy.run.ts` — entry point + +```ts +import alchemy from 'alchemy'; + +const app = await alchemy('nestri', { + password: process.env.ALCHEMY_PASSWORD // required for secrets +}); + +// Import infra modules in dependency order (SST-style) +await import('./infra/stage.ts'); +await import('./infra/secret.ts'); +await import('./infra/auth.ts'); +await import('./infra/api.ts'); + +await app.finalize(); +``` + +Key rules: + +- `alchemy(appName, opts)` creates a **scope** — resources register into this scope automatically +- `app.finalize()` must be called at the end to persist state +- Import order matters — resources that depend on others must be imported after +- `--dev` flag runs locally via Miniflare; omit it to deploy to Cloudflare + +## Infra resources + +Each resource is imported from `alchemy/cloudflare` and called with an ID + props: + +```ts +import { Worker, KVNamespace, D1Database } from 'alchemy/cloudflare'; + +export const kv = await KVNamespace('my-kv'); +export const db = await D1Database('my-db'); + +export const worker = await Worker('my-worker', { + entrypoint: 'apps/some-app/src/index.ts', + compatibility: 'node', // enables nodejs_compat flag + url: true, // assign workers.dev URL + bindings: { + KV: kv, // resource binding → KVNamespace at runtime + DB: db, // → D1Database + PLAIN_VAR: 'hello' // → plain_text binding + } +}); +``` + +### Supported resources (subset) + +| Resource | Import | Purpose | +| ------------- | -------------------- | ----------------------------------------------- | +| `Worker` | `alchemy/cloudflare` | Cloudflare Worker (entrypoint or inline script) | +| `KVNamespace` | `alchemy/cloudflare` | KV storage | +| `D1Database` | `alchemy/cloudflare` | D1 SQL database | +| `R2Bucket` | `alchemy/cloudflare` | R2 object storage | +| `Queue` | `alchemy/cloudflare` | Queue/pub-sub | + +### Compatibility flag + +Always add `compatibility: 'node'` to Workers that use Node.js built-ins (`node:async_hooks`, `crypto`, `node:stream`, etc.): + +```ts +Worker('api', { + entrypoint: 'apps/api/app/index.ts', + compatibility: 'node' // enables nodejs_compat +}); +``` + +## Stage detection + +```ts +// infra/stage.ts +import { Scope } from 'alchemy'; +const scope = Scope.getCurrentScope(); +export const stage = scope?.stage ?? 'dev'; +export const isPermanent = ['production', 'dev'].includes(stage); +``` + +Use stage for conditional infrastructure: + +```ts +const api = await Worker('api', { + ...(isPermanent && { + observability: { enabled: true }, + logpush: true + }) +}); +``` + +Pass `--stage` flag at runtime: `bun alchemy.run.ts --stage production` + +## Secrets and environment variables + +Three levels of env management, from most-secure to least: + +### 1. `alchemy.secret.env.X` (preferred) + +```ts +// infra/secret.ts +import alchemy from 'alchemy'; + +export const secret = { + steamApiKey: alchemy.secret.env.STEAM_API_KEY // reads process.env at deploy time + // Equivalent to: + // steamApiKey: alchemy.secret(process.env.STEAM_API_KEY), +}; +``` + +- Reads from `process.env` at deploy time +- Throws a descriptive error if the env var is missing +- Encrypted in Alchemy state files (`.alchemy/`) +- Deployed as `secret_text` binding (hidden from Cloudflare API) + +### 2. `alchemy.env()` (non-secret config) + +```ts +export const frontendUrl = alchemy.env('FRONTEND_URL', 'http://localhost:5173'); +``` + +- Optional default value +- Plain text — not encrypted +- Deployed as `plain_text` binding + +### 3. Plain strings in `bindings` (inline) + +```ts +bindings: { + MY_VAR: 'hello'; +} +``` + +- Hard-coded, visible in state files +- Deployed as `plain_text` binding + +### How bindings map to runtime types + +| Alchemy binding type | Deployed as | Runtime type | +| -------------------- | -------------- | -------------------------- | +| `Worker` | `service` | `Service` (has `.fetch()`) | +| `KVNamespace` | `kv_namespace` | `KVNamespace` | +| `D1Database` | `d1` | `D1Database` | +| `alchemy.secret()` | `secret_text` | `string` | +| plain `string` | `plain_text` | `string` | +| `Json(...)` | `json` | `typeof json` | + +## Service bindings (Worker → Worker) + +Pass one Worker as a binding to another: + +```ts +// infra/auth.ts +export const auth = await Worker('auth', { + entrypoint: 'apps/auth/src/index.ts', + compatibility: 'node', + bindings: { ... }, +}); + +// infra/api.ts +import { auth } from './auth.ts'; +export const api = await Worker('api', { + entrypoint: 'apps/api/app/index.ts', + bindings: { AUTH: auth }, +}); +``` + +At runtime, `env.AUTH` is a `Service` — call it directly: + +```ts +const response = await env.AUTH.fetch(request); +``` + +### OpenAuth client + service binding + +The `@openauthjs/openauth/client` only accepts a URL string for `issuer`, so use a custom `fetch` to route through the service binding: + +```ts +function getClient(env: Record) { + return createClient({ + issuer: 'https://auth.internal', // dummy — used for path construction + clientID: 'api', + fetch: (input, init) => { + const url = new URL(typeof input === 'string' ? input : input.url); + const request = new Request(url.pathname + url.search, init); + return (env.AUTH as { fetch: typeof fetch }).fetch(request); + } + }); +} +``` + +## Env propagation to Workers + +CF Workers receive env vars as the second argument to the `fetch` handler (`env`), NOT via `process.env`. Bridge the gap with a lazy + overridable schema: + +```ts +// packages/core/src/env.ts +import { memo } from '../utils/memo.ts'; + +let _overrides: Record = {}; + +export namespace Env { + export const Info = z.object({ + FRONTEND_URL: z.string().optional(), + STEAM_API_KEY: z.string().optional(), + AUTH_ISSUER_URL: z.string().optional() + }); + export type Info = z.infer; + + const _get = memo(() => Info.parse({ ...process.env, ..._overrides })); + + export function get(): Info { + return _get(); + } + + export function init(bindings: Record) { + _overrides = bindings; + _get.reset(); + } +} +``` + +Wire in the Hono entrypoint: + +```ts +export default { + fetch(request, env, ctx) { + Env.init(env); // merge CF bindings into Env + return app.fetch(request, env, ctx); + } +}; +``` + +Now any module that imports `Env.get()` gets the correct values — on Bun dev `process.env` provides them, on CF Workers the bindings override. + +## CLI usage + +```sh +# Local dev (Miniflare) +bun alchemy.run.ts --dev + +# Deploy to Cloudflare +bun alchemy.run.ts --stage production + +# Destroy all resources +bun alchemy.run.ts --destroy + +# With custom stage +bun alchemy.run.ts --stage wanjohiryan + +# Password (for encrypting secrets) +export ALCHEMY_PASSWORD="some-passphrase" +``` + +When deploying, set `CLOUDFLARE_API_TOKEN` or configure `alchemy login`. + +## Common patterns + +### Conditional infra per-stage + +```ts +Worker('api', { + ...(isPermanent && { logpush: true }), + ...(stage === 'production' && { scaling: { min: 3, max: 10 } }) +}); +``` + +### Across-app resource references + +Alchemy uses top-level await in infra files — resources resolve at import time within the active scope. The scope propagates via `AsyncLocalStorage`, so any `await import()` after `alchemy(appName)` picks it up. + +### .alchemy/ directory + +Created automatically — contains Miniflare state, build output, and encrypted state files. Add to `.gitignore`. + +```gitignore +.alchemy/ +``` + +--- + +### Index Rule: Null-Safe Exclusions (`IS DISTINCT FROM`) + +When writing indices to track data drift, synchronization deltas, or pending background worker states where values might be nullable, **always build a partial index utilizing Postgres-native `IS DISTINCT FROM`**. + +Standard inequality operators (`!=` or `<>`) evaluate to `NULL` if either column is `NULL`, causing them to bypass standard `WHERE` index filters. Using `is distinct from` allows Postgres to treat `NULL` as a real value for state comparison: + +- Excludes perfectly synchronized records completely from the index footprint. +- Optimizes heavy background worker poll queries directly into small, lightning-fast index scans. + +2. Add to the Pattern: index.ts (Domain Namespace) section + +Replace the existing create block and add the upsert block inside SomeModule: + +```ts +// ── create ─────────────────────────────────────────────────────────── +// Use Info.pick({…}) for the schema — keeps fields in sync with Info. +// Always use .returning() to get the updated row context in one database trip. +export const create = fn(Info.pick({ id: true, name: true, email: true }), async (input) => { + return Database.use(async (tx) => { + const [row] = await tx + .insert(SomeTable) + .values({ + id: input.id, + name: input.name, + email: input.email ?? null + }) + .returning(); + return row; + }); +}); + +// ── upsert ─────────────────────────────────────────────────────────── +// Simple copies use the input values directly. For coalesce-style set +// expressions, reference the excluded pseudo-table with unqualified +// identifiers: sql`excluded.${sql.identifier(SomeTable.name.name)}` — +// interpolating a column object (or its .name string) is invalid. +export const upsert = fn(Info.pick({ id: true, name: true }), async (input) => { + return Database.use(async (tx) => { + const [row] = await tx + .insert(SomeTable) + .values({ id: input.id, name: input.name }) + .onConflictDoUpdate({ + target: SomeTable.id, + set: { name: input.name } + }) + .returning(); + return row; + }); +}); +``` diff --git a/packages/core/CLAUDE.md b/packages/core/CLAUDE.md new file mode 100644 index 00000000..d7943f32 --- /dev/null +++ b/packages/core/CLAUDE.md @@ -0,0 +1,694 @@ +# `packages/core` — domain modules + +## Structure + +Every domain module lives in `packages/core/src//` as either a top-level namespace or a nested sub-module: + +``` +src// + ├── .sql.ts # (optional) Drizzle table for the parent entity + ├── index.ts # Parent namespace (e.g. User, Game, Team) + ├── .sql.ts # Sub-module table (e.g. fingerprint.sql.ts) + └── .ts # Sub-module namespace (e.g. export namespace Fingerprint) +``` + +### Sub-modules nested under parents + +| File | Namespace | Why | +| ----------------------- | --------------- | ------------------------------------- | +| `user/linked-account.*` | `LinkedAccount` | A user's OAuth/gaming identities | +| `user/fingerprint.*` | `Fingerprint` | SSH key fingerprints | +| `game/download.*` | `GameDownload` | Per-host game depot downloads | +| `user/library.*` | `Library` | User's owned games with playtime | +| `team/member.*` | `Member` | Team membership with role | +| `game/depot.*` | `Depot` | Platform-specific game content depots | + +Existing top-level modules: `user/`, `team/`, `game/`, `pairing-code/`, `steam/`, `auth/`, `db/`. + +Modules that don't own their own table (like `steam/`) only need a single `index.ts` exposing reusable `fn()` functions — no `.sql.ts` file. + +## Pattern: `.sql.ts` (Drizzle Table) + +```ts +// src//.sql.ts +import { pgTable, text, boolean, jsonb, uniqueIndex, index, pgEnum } from 'drizzle-orm/pg-core'; + +import { id, timestamps, ulid } from '../db/types.js'; + +// Enum (only if needed — co-located with its table) +export const SomeEnum = pgEnum('some_enum', ['a', 'b']); + +// FK imports — use the sql.ts files, never the index.ts (avoids circular deps) +import { UserTable } from '../user/user.sql.js'; + +export const SomeTable = pgTable( + 'some_table', + { + ...id, // char(30) PK, prefix: som_ + ...timestamps, // time_created, time_updated, time_deleted (all utc) + + // FK column — always use ulid() + .references() + userId: ulid('user_id') + .notNull() + .references(() => UserTable.id, { onDelete: 'cascade' }), + + // Scalar columns + name: text('name').notNull(), + email: text('email'), // nullable = omit .notNull() + flag: boolean('flag').notNull().default(false), + metadata: jsonb('metadata').$type<{}>(), // JSON blob + + // Enum column + provider: SomeEnum('provider').notNull() + }, + (t) => [ + uniqueIndex('some_table_provider_unique').on(t.provider, t.providerAccountId), + index('some_table_sync_idx') + .on(t.userId) + .where(sql`${t.localValue} is distinct from ${t.remoteValue}`), + index('some_table_user_idx').on(t.userId) + ] +); +``` + +### DB types (`src/db/types.ts`) + +| Helper | Output | +| ------------ | --------------------------------------------------------------- | +| `ulid(name)` | `char(30)` — for PKs and FKs | +| `id` | `{ id: ulid('id').primaryKey().notNull() }` — spread as `...id` | +| `utc(name)` | `timestamp with time zone` | +| `timestamps` | `{ timeCreated, timeUpdated (auto), timeDeleted }` | + +### Naming conventions + +- Table name: `snake_case` (e.g. `linked_account`, `team_member`) +- Column name: `snake_case` (e.g. `user_id`, `provider_account_id`, `time_created`) +- TypeScript field names: `camelCase` matching the column (drizzle maps them) +- Index names: `{table}_{column(s)}_unique` / `{table}_{column}_idx` + +--- + +## Pattern: `index.ts` (Domain Namespace) + +```ts +// src//index.ts +import { eq, and, isNull, sql } from 'drizzle-orm'; +import z from 'zod'; + +import { Database } from '../db/index.js'; +import { Examples } from '../examples.js'; +import { fn } from '../fn.js'; +import { SomeTable, SomeEnum } from './.sql.js'; + +export namespace SomeModule { + // ── Info schema ───────────────────────────────────────────────────── + // Single source of truth for the entity shape. + // Every field typed here; .meta() adds OpenAPI metadata. + // When a field changes here, TypeScript catches every usage. + export const Info = z + .object({ + id: z.string().meta({ + description: '…', + example: Examples.SomeModule.id, + }), + // For enum fields, use z.enum(SomeEnum.enumValues) to stay in sync: + provider: z.enum(SomeEnum.enumValues).meta({ … }), + // Nullable + optional for JSON-blob / optional fields: + metadata: z.record(z.string(), z.unknown()).nullable().optional().meta({ … }), + }) + .meta({ + ref: 'SomeModule', + description: '…', + example: Examples.SomeModule, + }); + + export type Info = z.infer; + + // ── create ─────────────────────────────────────────────────────────── + // Use Info.pick({…}) for the schema — keeps fields in sync with Info. + // Input is the parsed object. + // Use Database.use() for single-operation writes. + export const create = fn( + Info.pick({ id: true, name: true, email: true }), + async (input) => { + await Database.use(async (tx) => { + await tx.insert(SomeTable).values({ + id: input.id, + name: input.name, + email: input.email ?? null, + }); + }); + return input.id; + } + ); + + // ── Single-field lookups ───────────────────────────────────────────── + // Use Info.shape. for the schema. + // The callback receives the raw value, not { field: value }. + export const fromID = fn(Info.shape.id, async (id) => { + return Database.use(async (tx) => { + return tx + .select() + .from(SomeTable) + .where(and(eq(SomeTable.id, id), isNull(SomeTable.timeDeleted))) + .then((rows) => rows.at(0) ?? null); + }); + }); + + export const fromSlug = fn(Info.shape.slug, async (slug) => { + // … + }); + + // ── Multi-field lookups ────────────────────────────────────────────── + // Use Info.pick({ field1: true, field2: true }) + export const findByProvider = fn( + Info.pick({ provider: true, providerAccountId: true }), + async (input) => { + return Database.use(async (tx) => { + return tx + .select() + .from(SomeTable) + .where(and( + eq(SomeTable.provider, input.provider), + eq(SomeTable.providerAccountId, input.providerAccountId), + isNull(SomeTable.timeDeleted), + )) + .then((rows) => rows.at(0) ?? null); + }); + } + ); + + // ── List (no args) ─────────────────────────────────────────────────── + // Plain async function — no fn() wrapper since there's no input. + export async function list() { + return Database.use(async (tx) => { + return tx + .select() + .from(SomeTable) + .where(isNull(SomeTable.timeDeleted)) + .orderBy(SomeTable.timeCreated); + }); + } + + // ── List by FK ─────────────────────────────────────────────────────── + export const listByUser = fn(Info.shape.userId, async (userId) => { + return Database.use(async (tx) => { + return tx + .select() + .from(SomeTable) + .where(and(eq(SomeTable.userId, userId), isNull(SomeTable.timeDeleted))) + .orderBy(SomeTable.timeCreated); + }); + }); + + // ── Update ─────────────────────────────────────────────────────────── + // If the update uses Info fields + possibly extra fields: + export const updateSomething = fn( + Info.pick({ id: true, otherField: true }).extend({ + extraField: z.string(), + }), + async (input) => { + await Database.use(async (tx) => { + await tx + .update(SomeTable) + .set({ otherField: input.otherField }) + .where(eq(SomeTable.id, input.id)); + }); + } + ); + + // ── Soft-delete ────────────────────────────────────────────────────── + // Use sql\`now()\` for the timestamp (consistent with DB time). + export const remove = fn(Info.shape.id, async (id) => { + await Database.use(async (tx) => { + await tx + .update(SomeTable) + .set({ timeDeleted: sql`now()` }) + .where(eq(SomeTable.id, id)); + }); + }); + + // ── serialize ──────────────────────────────────────────────────────── + // Converts a DB row into the public API shape. Keeps serialization + // decoupled from the DB layer. + export function serialize(input: typeof SomeTable.$inferSelect): z.infer { + return { + id: input.id, + name: input.name, + // Cast enums since drizzle returns a string at runtime: + provider: input.provider as Info['provider'], + }; + } + + // ── listByUserWithGames ────────────────────────────────────────────── + // JOIN query with serialization inside the fn() boundary. + // The .then() chain maps raw rows to the public shape immediately. + export const listByUserWithGames = fn(Info.shape.userId, async (userId) => { + return Database.use(async (tx) => { + return tx + .select({ + library: UserLibraryTable, + game: GameTable + }) + .from(UserLibraryTable) + .leftJoin(GameTable, eq(UserLibraryTable.gameId, GameTable.id)) + .where(and(eq(UserLibraryTable.userId, userId), isNull(UserLibraryTable.timeDeleted))) + .orderBy(UserLibraryTable.timeCreated) + .then((rows) => + rows + .filter((row) => row.game !== null) + .map((row) => ({ + id: row.library.id, + game: Game.serialize(row.game!), + playtime2w: row.library.playtime2w, + playtimeForever: row.library.playtimeForever, + lastPlayed: row.library.lastPlayed?.toISOString() ?? null + })) + ); + }); + }); +} +``` + +### Key rules for `fn()` usage + +| Case | Schema | Callback receives | +| -------------------- | ------------------------------------------ | ------------------------------------- | +| Single field | `Info.shape.field` | Raw value (`string`, `boolean`, etc.) | +| Multiple fields | `Info.pick({a:true, b:true})` | `{ a, b }` object | +| Full entity | `Info` | Full `Info` object | +| Info fields + extras | `Info.pick({…}).extend({extra: z.type()})` | `{ …fields, extra }` | +| No input | Regular `async function` | n/a | + +### What `fn()` does + +```ts +fn(schema, callback); +// → (input) => { schema.parse(input); return callback(parsed); } +// The returned function also has a .schema property for OpenAPI introspection. +``` + +--- + +## Serialization Boundary Rule + +**All data transformation — including JOIN deserialization, field mapping, date stringification, and null filtering — happens INSIDE the `fn()` boundary, right after the DB query in a `.then()` chain. The API route is a dumb pass-through.** + +### Why this matters + +| Approach | Queries | Boundary clarity | +| ----------------------------------------------------- | ------------ | -------------------------------------- | +| **Bad**: Raw rows from core, map/filter in route | N+1 (or raw) | Leaky — route knows DB schema | +| **Bad**: JOIN in core, but serialize in route | 1 | Still leaky — route owns shaping logic | +| **Good**: JOIN + serialize in `.then()` inside `fn()` | 1 | Clean — core returns JSON-safe objects | + +### The pattern + +```ts +// GOOD: serialization inside fn() +export const listByUserWithGames = fn(Info.shape.userId, async (userId) => { + return Database.use(async (tx) => { + return tx + .select({ library: UserLibraryTable, game: GameTable }) + .from(UserLibraryTable) + .leftJoin(GameTable, eq(UserLibraryTable.gameId, GameTable.id)) + .where(...) + .orderBy(...) + .then((rows) => + rows + .filter((row) => row.game !== null) + .map((row) => ({ + id: row.library.id, + game: Game.serialize(row.game!), + lastPlayed: row.library.lastPlayed?.toISOString() ?? null + })) + ); + }); +}); + +// GOOD: API route is a thin pass-through +async (c) => { + const data = await Library.listByUserWithGames(Actor.userID); + return c.json({ data }); +} +``` + +### Rules + +1. **Never let raw Drizzle `$inferSelect` rows escape the core module.** If a function returns joined data, it must be shaped before the return. +2. **Use `.then()` after the query for map/filter/serialize.** Keeps the async pipeline declarative and co-located with the SQL. +3. **Re-use sibling `serialize()` functions for joined tables.** e.g. `Game.serialize(row.game!)` when joining `GameTable`. +4. **API routes only do:** auth checks, input validation, calling the core fn, and `c.json({ data })`. No `.map()`, no `.filter()`, no field remapping. + +--- + +### Mutation Rule: Single-Query Reads via `.returning()` + +Never execute a separate `tx.select()` or trigger a lookup function immediately after an `insert` or `upsert` mutation to fetch the updated state of a row. + +Postgres natively supports the `RETURNING` clause. Always append `.returning()` directly to your mutation chains and destructure the resulting array (`const [row] = await tx...`). This ensures mutations remain atomic, avoids unnecessary connection pool overhead, and removes the latency penalty of running two sequential database operations. + +--- + +## Pattern: ID generation (`src/id.ts`) + +```ts +Identifier.ascending('user') // → "usr_" +Identifier.ascending('team') // → "tem_" +Identifier.ascending('linkedAccount') // → "lac_" + +// Prefixes are defined in Identifier.prefixes: +{ + user: 'usr', + linkedAccount: 'lac', + team: 'tem', + teamMember: 'mem', + verification: 'ver', +} +``` + +The IDs are 30-char strings: `{prefix}_{26 base62 chars}`. They are monotonically increasing (time-sortable) when using `ascending()`. + +--- + +## Pattern: Examples (`src/examples.ts`) + +```ts +export namespace Examples { + export const Id = (prefix: keyof typeof Identifier.prefixes) => + `${Identifier.prefixes[prefix]}_XXXXXXXXXXXXXXXXXXXXXXXXX`; + + export const User = { id: Id('user'), name: '…', email: '…', … }; + export const LinkedAccount = { id: Id('linkedAccount'), provider: 'steam', … }; + export const Team = { id: Id('team'), slug: 'my-team', … }; + export const Member = { id: Id('teamMember'), role: 'owner' as const, … }; + export const Fingerprint = { id: Id('userFingerprint'), fingerprint: '…', … }; + export const GameDownload = { id: Id('gameDownload'), hostId: 'hst_…', gameId: Id('game'), status: 'downloading', … }; + export const Library = { id: Id('userLibrary'), playtimeForever: 150000, … }; + export const Depot = { id: Id('gameDepot'), depotId: 730, … }; +} +``` + +Every entity in `Examples` must be added and imported by the `Info` schema's `.meta({ example: … })`. + +--- + +## Pattern: Environment (`src/env.ts`) + +```ts +export namespace Env { + export const Info = z.object({ + NODE_ENV: z.enum(['development', 'production', 'test']).default('development'), + FRONTEND_URL: z.string().optional(), + STEAM_API_KEY: z.string().optional(), + AUTH_ISSUER_URL: z.string().optional() // used by API auth middleware to verify tokens + }); + export type Info = z.infer; + export const env: Info = Info.parse(process.env); +} +``` + +--- + +## Pattern: OpenAuth Subjects (`src/auth/subjects.ts`) + +```ts +import { createSubjects } from '@nestri/auth/subject'; +import { z } from 'zod'; + +export const subjects = createSubjects({ + user: z.object({ + userID: z.string(), + linkedAccountID: z.string() + }) +}); +``` + +The JWT contains `{ type: 'user', properties: { userID, linkedAccountID } }`. +The API verifies via `client.verify(subjects, token)` from `@nestri/auth/client`. + +--- + +## Entity relationships + +``` +User ───1:N─── LinkedAccount ← auth methods (Steam, Epic, etc.) + │ + ├─── 1:N ─── Fingerprint ← SSH public keys + ├─── 1:N ─── Library ← owned games with playtime + │ + └─── N:M ─── Team ← via Member + └─ role: owner | admin | member + +Game ───1:N─── Download ← per-host game depot downloads +``` + +- **User**: Person record. Email is nullable (gaming accounts don't provide one). +- **LinkedAccount**: A gaming/OAuth identity. `(provider, providerAccountId)` is unique. +- **Team**: Organization for billing/collaboration. First team is auto-created as "personal" team. +- **TeamMember**: Joins User → Team with a role. `(teamId, userId)` is unique. + +--- + +## Database access + +```ts +// Auto-scoped transaction (creates one if outside a transaction): +await Database.use(async (tx) => { + await tx.insert(SomeTable).values({ … }); + const result = await tx.select().from(SomeTable).where(…); +}); + +// Explicit transaction: +await Database.transaction(async (tx) => { + // All operations in one atomic transaction +}); + +// Side-effect queued after transaction commits: +Database.effect(() => sendEmail(…)); +``` + +--- + +## Soft-delete convention + +Every table has `time_deleted` (nullable timestamp). Queries filter with `isNull(table.timeDeleted)`. Deletion sets `timeDeleted: sql\`now()\`` — never hard-deletes. + +--- + +## Dependency rule + +- `.sql.ts` files may import from other `.sql.ts` files (for FK references). +- `index.ts` files may import from other `index.ts` files and `.sql.ts` files. +- Never import an `index.ts` from within a `.sql.ts` — that creates circular deps. + +--- + +## Actor Model (`packages/core/src/actor.ts`) + +The Actor model identifies who/what is making a request. It uses `AsyncLocalStorage` (via `Context.create()`) so the actor is accessible anywhere in the call chain without passing it around. + +### Actor types + +```ts +type ActorInfo = + | { type: 'public'; properties: {} } + | { type: 'user'; properties: { userID: string; linkedAccountID: string } } + | { + type: 'member'; + properties: { userID: string; teamID: string; role: 'owner' | 'admin' | 'member' }; + } + | { type: 'system'; properties: { teamID: string } } + | { type: 'admin'; properties: {} }; +``` + +### API + +```ts +Actor.use(); // → ActorInfo (throws if no context set) +Actor.with(value, fn); // Run fn in the given actor context +Actor.assert(type); // Assert current actor type, returns narrowed type +Actor.type; // → 'public' | 'user' | 'member' | 'system' | 'admin' +Actor.userID; // → string (user/member only) +Actor.linkedAccountID; // → string (user only) +Actor.useTeam; // → string (member/system only — the teamID) +Actor.role; // → 'owner' | 'admin' | 'member' (member only) +Actor.isSignedIn; // → boolean (true if not public) +``` + +### When to pull from Actor vs pass as param + +Functions that create resources owned by the current user (e.g. `Team.create`) pull `userID` from the actor context rather than requiring it as a parameter. This avoids passing `ownerId`/`userId` through the entire call chain. + +Domain functions that need the actor's identity import `Actor` and call `Actor.userID` inside the `fn()` callback: + +```ts +export const create = fn(Info.pick({ id: true, name: true, slug: true }), async (input) => { + const ownerId = Actor.userID; // from AsyncLocalStorage + // ... +}); +``` + +### Actor.userID inside Database.transaction() + +When doing find-or-create logic that must scope to the authenticated user, pull `Actor.userID` **inside** the `Database.transaction()` callback. This keeps scoping co-located with the DB logic: + +```ts +export const link = fn( + z.object({ steamId: z.string(), profile: z.record(z.string(), z.unknown()).optional() }), + async (input) => { + return Database.transaction(async () => { + const existing = await LinkedAccount.findByProvider({ ... }); + if (existing) return existing.id; + const id = Identifier.ascending('linkedAccount'); + await LinkedAccount.create({ id, userId: Actor.userID, provider: 'steam', ... }); + return id; + }); + } +); +``` + +This ensures the API endpoint is scoped to the user only — no `userId` param is passed from the route handler. The `Actor.userID` is set by the auth middleware's `Actor.with()`, propagated via `AsyncLocalStorage`. + +Callers must wrap actor-dependent work in `Actor.with()` before calling these functions. The API middleware does this automatically for HTTP requests. The auth worker sets it up explicitly: + +```ts +await Actor.with({ type: 'user', properties: { userID, linkedAccountID } }, async () => { + await Team.createPersonal({ displayName: personaname }); +}); +``` + +--- + +## Auth Flow + +### Auth Worker (`apps/auth/src/index.ts`) + +A Cloudflare Worker using `@nestri/auth` (OpenAuth). Entry point is the `success` callback after OAuth: + +1. Steam returns `steamid` → worker fetches profile from Steam API +2. Inside `Database.transaction()`: looks up existing `LinkedAccount.findByProvider`; if found returns existing user, else creates `User` + `LinkedAccount` +3. Wraps post-login setup in `Actor.with()`, then checks `Member.listByUser(userID)` +4. If no memberships, calls `Team.createPersonal({ displayName })` +5. Issues JWT via `context.subject('user', { userID, linkedAccountID })` + +### Admin Auth via Shared Secret + +Server-to-server calls can authenticate as an `admin` actor by setting the `x-nestri-admin-token` header to the value of `ADMIN_SHARED_SECRET`. This bypasses JWT auth entirely and grants a system-level actor with no user scope — useful for operations like adding games to the DB, syncing data, or other admin tasks. + +Configure `ADMIN_SHARED_SECRET` in `.env`; defaults to the same value as `SSH_AUTH_KEY` in dev (`dev-ssh-auth-key-change-in-prod`). + +### Auth Middleware (`apps/api/app/middleware/auth.ts`) + +Hono middleware that runs on every API request: + +1. Checks `x-nestri-admin-token` header — if it matches `Env.get().ADMIN_SHARED_SECRET`, sets actor to `admin` and proceeds immediately +2. Otherwise, reads `Authorization: Bearer ` header +3. Verifies via `client.verify(subjects, token)` from `@nestri/auth/client` +4. If valid `user` subject: + - Checks `x-nestri-team` header for team-scoped access + - If team header present, verifies membership via `Member.findByTeamAndUser` + - Sets actor to `member` (with role) or `user` type +5. If no token/invalid: sets actor to `public` type +6. Exports `notPublic` guard middleware — throws `VisibleError('authentication', UNAUTHORIZED, …)` if actor is `public`. Caught by `onError` → 401 JSON response. The `admin` actor passes this guard (it's not `public`), so admin routes can use `.use(notPublic)` like any other protected route. + +### OpenAuth Subjects (`src/auth/subjects.ts`) + +```ts +export const subjects = createSubjects({ + user: z.object({ userID: z.string(), linkedAccountID: z.string() }) +}); +``` + +The JWT contains `{ type: 'user', properties: { userID, linkedAccountID } }`. +Env var `AUTH_ISSUER_URL` configures which issuer to trust for token verification. + +--- + +## Team Discriminator System + +When creating a personal team (`Team.createPersonal`), the slug is derived from the display name: + +```ts +// 1. Sanitize: lowercase, replace non-alphanumeric with dashes, trim edges, max 50 chars +const baseSlug = displayName.toLowerCase().replace(/[^a-z0-9]+/g, '-')... + +// 2. Check if slug exists (fromSlug) +// 3. If taken, append random 4-digit discriminator: "name-1234" +const slug = existing ? `${baseSlug}-${discriminator}` : baseSlug; +``` + +This mirrors Discord's discriminator pattern. The discriminator is part of the slug string — not a separate column. + +--- + +## `fn()` and Actor Context + +`fn()` itself is unchanged — it validates input against a zod schema and calls the callback. The actor context is available inside `fn()` callbacks because `Actor.with()` uses `AsyncLocalStorage`, which propagates automatically through `await` chains. + +Every `fn()` callback can import and use `Actor` directly. No special wiring needed. + +--- + +## API Error Pattern (`packages/core/src/error.ts`) + +Centralized error types used by both the domain layer and the API. + +### ErrorResponse + +Zod schema for OpenAPI error responses: + +```ts +import { z } from 'zod'; + +export const ErrorResponse = z + .object({ + type: z.enum([ + 'validation', + 'authentication', + 'forbidden', + 'not_found', + 'already_exists', + 'rate_limit', + 'internal' + ]), + code: z.string(), + message: z.string(), + param: z.string().optional(), + details: z.any().optional() + }) + .meta({ ref: 'ErrorResponse' }); +``` + +### ErrorCodes + +Structured error code constants: + +| Category | Codes | +| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `Validation` | `MISSING_REQUIRED_FIELD`, `ALREADY_EXISTS`, `TEAM_ALREADY_EXISTS`, `INVALID_PARAMETER`, `INVALID_FORMAT`, `INVALID_STATE`, `IN_USE` | +| `Authentication` | `UNAUTHORIZED`, `INVALID_TOKEN`, `EXPIRED_TOKEN`, `INVALID_CREDENTIALS` | +| `Permission` | `FORBIDDEN`, `INSUFFICIENT_PERMISSIONS`, `ACCOUNT_RESTRICTED` | +| `NotFound` | `RESOURCE_NOT_FOUND` | +| `RateLimit` | `TOO_MANY_REQUESTS`, `QUOTA_EXCEEDED` | +| `Server` | `INTERNAL_ERROR`, `SERVICE_UNAVAILABLE`, `DEPENDENCY_FAILURE` | + +### VisibleError + +Throw this for any user-facing error. It carries structured data and converts cleanly to HTTP: + +```ts +throw new VisibleError( + 'not_found', + ErrorCodes.NotFound.RESOURCE_NOT_FOUND, + `User ${id} does not exist` +); +``` + +- `.statusCode()` — maps `type` → HTTP status (validation→400, authentication→401, forbidden→403, not_found→404, already_exists→409, rate_limit→429, internal→500) +- `.toResponse()` → `{ type, code, message, param?, details? }` + +The API's global `onError` handler catches `VisibleError` + `HTTPException` + unknown errors, logging each and returning the correct JSON shape. + +---