mirror of
https://github.com/nestriness/nestri.git
synced 2026-09-19 17:25:19 +03:00
ci: a merge to prod releases the control plane, migrator included
`release-prod.yml` for api and auth, mirroring the edge's. The gate is not "you may not merge" — it is that merging does not deploy: tests run against a real Postgres, both binaries are executed, and a failure anywhere means no release exists, so the machine keeps serving what it has. The new piece is `nestri-migrate`, because the deploy runs migrations *before* it swaps a release into place and had nothing to run. `drizzle-kit migrate` reads the migrations folder at runtime, which is right on a laptop and wrong on a server: the deploy ships flat, checksummed files into bin/, and a migrator that needs a directory beside it can be pointed at the wrong directory. So the folder is baked into the binary — generated on every build rather than committed, so it cannot drift — and the artefact's checksum then covers every statement it will run. It reimplements drizzle's bookkeeping in thirty lines of SQL rather than calling into `db.dialect.migrate`: same table, same schema, same sha256 over the whole file, same high-water-mark comparison. That equivalence is the one thing that must not rot, because the production database was first migrated by drizzle-kit and a disagreement means a migration applied twice. Two checks hold it: CI applies the migrations with drizzle-kit and then asserts the embedded set reports nothing pending, and the hashes were verified by hand against the live database — all fourteen match to the byte. Proven before shipping, against the real database: nothing pending on the deployed schema, 14 unchanged rows, and a scratch database migrated from empty to the same 22 tables and the same high-water mark, idempotent on a second run. Refuses with exit 2 when DATABASE_URL is absent rather than defaulting to localhost, which would be a migrator reporting success having migrated nothing. setup-bun is pinned to a commit and not to `v2`. A moving major tag is fine everywhere else in this repository; this workflow is the only thing between a merge and a process serving users, and there is no first-party bun action to prefer instead.
This commit is contained in:
56
packages/core/scripts/embed-migrations.ts
Normal file
56
packages/core/scripts/embed-migrations.ts
Normal file
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* Bake `migrations/` into a TypeScript module so the migrator can be a single
|
||||
* self-contained binary.
|
||||
*
|
||||
* `drizzle-kit migrate` reads the folder at runtime, which is the right shape
|
||||
* on a laptop and the wrong one on a server: the deploy agent ships flat,
|
||||
* checksummed files into `bin/`, and a migrator that needs a directory beside
|
||||
* it is a migrator that can be run against the wrong directory. Embedding
|
||||
* removes the question — the binary *is* the migration set, and its checksum
|
||||
* covers every statement it will run.
|
||||
*
|
||||
* Run before `bun build --compile`. It runs on every build rather than being
|
||||
* committed, so the embedded set cannot drift from the folder.
|
||||
*
|
||||
* bun run packages/core/scripts/embed-migrations.ts
|
||||
*/
|
||||
import crypto from 'node:crypto';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
const here = path.dirname(new URL(import.meta.url).pathname);
|
||||
const folder = path.join(here, '..', 'migrations');
|
||||
const out = path.join(here, '..', 'src', 'migrations.generated.ts');
|
||||
|
||||
type JournalEntry = { idx: number; when: number; tag: string; breakpoints: boolean };
|
||||
|
||||
const journal = JSON.parse(
|
||||
fs.readFileSync(path.join(folder, 'meta', '_journal.json'), 'utf8')
|
||||
) as { entries: JournalEntry[] };
|
||||
|
||||
// Deliberately identical to drizzle-orm's own `readMigrationFiles`: the same
|
||||
// `--> statement-breakpoint` split, the same sha256 over the whole file, the
|
||||
// same `when` as the ordering key. The live database was migrated by
|
||||
// drizzle-kit, so anything else here would re-apply migrations that have
|
||||
// already run.
|
||||
const migrations = journal.entries.map((e) => {
|
||||
const sql = fs.readFileSync(path.join(folder, `${e.tag}.sql`), 'utf8');
|
||||
return {
|
||||
tag: e.tag,
|
||||
sql: sql.split('--> statement-breakpoint'),
|
||||
bps: e.breakpoints,
|
||||
folderMillis: e.when,
|
||||
hash: crypto.createHash('sha256').update(sql).digest('hex')
|
||||
};
|
||||
});
|
||||
|
||||
fs.writeFileSync(
|
||||
out,
|
||||
`// Generated by scripts/embed-migrations.ts. Do not edit.\n` +
|
||||
`// ${migrations.length} migrations, ${migrations.at(-1)?.tag ?? 'none'} last.\n` +
|
||||
`export type Embedded = {\n` +
|
||||
`\ttag: string;\n\tsql: string[];\n\tbps: boolean;\n\tfolderMillis: number;\n\thash: string;\n};\n\n` +
|
||||
`export const MIGRATIONS: Embedded[] = ${JSON.stringify(migrations, null, '\t')};\n`
|
||||
);
|
||||
|
||||
console.log(`embedded ${migrations.length} migrations -> ${path.relative(process.cwd(), out)}`);
|
||||
113
packages/core/src/migrate.ts
Normal file
113
packages/core/src/migrate.ts
Normal file
@@ -0,0 +1,113 @@
|
||||
/**
|
||||
* The migrator, as a binary.
|
||||
*
|
||||
* nestri-migrate apply everything pending
|
||||
* nestri-migrate --check say what is pending, change nothing
|
||||
*
|
||||
* The deploy agent runs this **before** it swaps the new release into place,
|
||||
* which is what makes the additive-only rule load-bearing: a rollback moves
|
||||
* code and never schema, so every migration has to be readable by the release
|
||||
* it is replacing. Add a column, deploy, stop writing the old one, drop it in
|
||||
* a later release — never in the same one.
|
||||
*
|
||||
* The bookkeeping is deliberately identical to `drizzle-orm`'s own
|
||||
* `PgDialect.migrate`, down to the table, the schema and the high-water-mark
|
||||
* comparison, because this database was first migrated by `drizzle-kit` and
|
||||
* both must agree about what has already run. Verified 2026-09-17 against the
|
||||
* live database: all fourteen hashes and timestamps match to the byte.
|
||||
*
|
||||
* It talks to Postgres directly rather than through `drizzle-orm` — this is
|
||||
* thirty lines of SQL, and the version that goes near production schema should
|
||||
* be the one you can read in full.
|
||||
*/
|
||||
import postgres from 'postgres';
|
||||
|
||||
import { MIGRATIONS } from './migrations.generated.js';
|
||||
|
||||
const SCHEMA = 'drizzle';
|
||||
const TABLE = '__drizzle_migrations';
|
||||
|
||||
async function main() {
|
||||
const check = process.argv.includes('--check');
|
||||
|
||||
const url = process.env.DATABASE_URL;
|
||||
if (!url) {
|
||||
console.error('nestri-migrate: DATABASE_URL is not set');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// One connection, and no idle timeout worth the name: this process exists
|
||||
// for a few seconds and then stops.
|
||||
const sql = postgres(url, {
|
||||
max: 1,
|
||||
idle_timeout: 5,
|
||||
connect_timeout: 30,
|
||||
// `CREATE ... IF NOT EXISTS` raises a NOTICE every single run, and
|
||||
// postgres-js prints the whole struct by default -- so the steady state
|
||||
// of this binary was eighteen lines of noise around one line of fact.
|
||||
// Errors are unaffected; they are thrown, not noticed.
|
||||
onnotice: () => {}
|
||||
});
|
||||
|
||||
try {
|
||||
await sql.unsafe(`CREATE SCHEMA IF NOT EXISTS "${SCHEMA}"`);
|
||||
await sql.unsafe(`
|
||||
CREATE TABLE IF NOT EXISTS "${SCHEMA}"."${TABLE}" (
|
||||
id SERIAL PRIMARY KEY,
|
||||
hash text NOT NULL,
|
||||
created_at bigint
|
||||
)
|
||||
`);
|
||||
|
||||
const rows = await sql.unsafe(
|
||||
`select id, hash, created_at from "${SCHEMA}"."${TABLE}" order by created_at desc limit 1`
|
||||
);
|
||||
const last = rows[0] ? Number(rows[0].created_at) : null;
|
||||
|
||||
// A high-water mark, not a set of hashes. That is drizzle's rule and
|
||||
// changing it here would make the two disagree about a migration that
|
||||
// was applied out of order.
|
||||
const pending = MIGRATIONS.filter((m) => last === null || last < m.folderMillis);
|
||||
|
||||
if (pending.length === 0) {
|
||||
console.log(`nestri-migrate: nothing to do (${MIGRATIONS.length} applied)`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (check) {
|
||||
console.log(`nestri-migrate: ${pending.length} pending`);
|
||||
for (const m of pending) console.log(` ${m.tag}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// All of them in one transaction, as drizzle does. Postgres has
|
||||
// transactional DDL, so a failure half way through leaves the schema
|
||||
// exactly as it was rather than half-migrated with a release about to
|
||||
// be swapped in on top of it.
|
||||
await sql.begin(async (tx) => {
|
||||
for (const m of pending) {
|
||||
console.log(`nestri-migrate: applying ${m.tag}`);
|
||||
for (const stmt of m.sql) {
|
||||
if (stmt.trim() === '') continue;
|
||||
await tx.unsafe(stmt);
|
||||
}
|
||||
await tx.unsafe(
|
||||
`insert into "${SCHEMA}"."${TABLE}" ("hash", "created_at") values($1, $2)`,
|
||||
[m.hash, m.folderMillis]
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
console.log(`nestri-migrate: applied ${pending.length}`);
|
||||
} finally {
|
||||
await sql.end();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
// Non-zero and loud. The agent refuses to swap the release in when this
|
||||
// fails, so the thing that matters most is that it cannot fail quietly.
|
||||
console.error('nestri-migrate: failed');
|
||||
console.error(err);
|
||||
process.exit(1);
|
||||
});
|
||||
Reference in New Issue
Block a user