Browse this guideDeploy and roll back
DevelopmentUpdated 2026-11-13

Deploy and roll back

Every green commit on main deploys. The order of a migration and the code that calls it is the part that goes wrong, so it comes first.

Migrations and code, in order

Apply a migration to the Site's hosted database before you deploy any code that calls it. The Site web app's access gate calls api.current_administrator() on every route and answers 503 without it. The Review Queue, the project page, and the worker's roster backfill each call functions their own migrations add. Migrations apply in filename order.

The MCP service calls api.current_principal() on every request. Apply 20261112120000_current_principal.sql to the hosted Site before deploying the MCP service that uses it; authenticated requests fail without it. Applying the migration before the code is safe: it adds a read-only function and changes no stored data.

  1. Push the migration first. From site/db, run supabase db push, then supabase migration list.
    • Check: the new file shows as applied remotely.
    • If not: read the push error. Do not edit an applied migration; write a new one.
  2. Then merge the code. A green CI run on main starts the deploys.

Do this for every Site, because each Site has its own database.

Deploy the backend

The Deploy workflow builds the backend image once per commit, pushes it to GitHub's container registry tagged with the full commit SHA, then points each Site's ingress and worker services at that tag and its mcp service when that Site configures one.

  1. Let it run. A push to main runs CI, and a green CI starts Deploy. Runs go one at a time, oldest first, so an older commit never deploys over a newer one.
    • Check: the run summary lists the commit, the image, and its digest, and each Site's deploy job finishes. The deploy waits for Railway to report success and then checks ingress's /health and, when configured, MCP's /health.
    • If not: the job's exit code says where it stopped: 2 a missing setting or bad tag, 3 updating the service, 4 deploying, 5 waiting, 6 the health check.
  2. Check the worker yourself. The worker has no healthcheck, so a green deploy proves only that Railway started it. Open the worker's logs a few minutes later.
    • Check: it is still running and has logged no error. A worker that exits with a Deposit sign-in error has a stale credential; see Rotate credentials.

Service settings (start command, healthcheck path, replicas, restart policy) come from site/backend/deploy/services.json, not the dashboard. A dashboard edit is overwritten on the next deploy. A rollback applies today's manifest to an older image.

Workflow: deploy.yml. Script: railway_deploy.py.

Deploy the web app

Deploy web builds the Site web app once per Site for each green main commit, because the build inlines that Site's public settings, and uploads it to the Site's Vercel project.

Workflow: deploy-web.yml.

Roll back the backend

  1. Find the last good full SHA. Use a commit whose Deploy run succeeded and whose image exists in the registry.
  2. Dispatch Deploy from main with that SHA as image_tag. It must be the full 40-character lowercase SHA. No image is rebuilt.
    • Check: the run summary shows build: false and the SHA you gave, and the health check passes.
    • If not: exit 2 means the tag is not a full SHA. A dispatch from any branch other than main is refused by the Site's GitHub Environment.
  3. Check the worker as above.

Roll back the web app

There is no rollback tag for the web app. Vercel keeps earlier deployments: promote the last good one from the Vercel dashboard, or dispatch Deploy web from main after reverting the bad commit.

Roll back a migration

Roll back in reverse: code first, then the migration.

  1. Roll the code back. Web: promote the last good deployment. Backend: dispatch Deploy with the last good SHA.
    • Check: neither is running code that calls the migration's functions.
  2. Run the migration's reverse script from site/db/rollback/ with psql, against the Site's operator connection string. It has the same filename as the migration it undoes. Nothing runs reverse scripts automatically, and the Supabase CLI's own migration down is not a substitute: it resets and replays the database, discarding every row written since.
    • Check: the objects the migration added are gone from the database itself. Do not rely on supabase migration list here, because it reads the CLI's history and not the reverse script's effect. Then confirm the Site's health endpoint still reports db: ok.
  3. For anything beyond the latest migration, use point-in-time recovery instead. It is the ordinary path, and it is an action you take against Supabase, not a script.

Read the reverse script's header first

Before you run any reverse script, read the comment at the top of that exact file. It says whether the script is safe against data written since, what it loses, and what must be rolled back first. The tables below summarize the headers as they stand; the file is authoritative, and a new migration's script may say more than this page does.

Read-only migrations roll back safely. They add reads and write no data. Rolling one back breaks the screen, gate, or service that calls it, so roll that code back first.

Other safe ones, with a condition.

MigrationCondition
ingress_roleSafe for data, but ingress loses its access at once. Point ingress's SITE_DATABASE_URL back at a privileged role first.
deposit_process_identitiesSafe, touches no Deposit object or Auth user. A Site rolled back here also needs the single earlier Deposit Auth user and the earlier backend release; the script creates and deletes no Auth users.
deposit_backend_auth_identitySafe; it only recreates a role, grants, and policies.

Reverse scripts that lose data or refuse. Each is a deliberate choice, not a routine undo.

MigrationWhat reversing it does
staff_roleRefuses while any active or revoked principal holds staff; reassign those principals first. Otherwise removes the role and project-people capability, helpers and policies, and restores administrator-only sender names without deleting other stored records.
deposit_storage_identityRefuses unless the Deposit bucket is empty, and never deletes a Deposit object. Once a Site has captured content, use point-in-time recovery. Run it inside a transaction and end it at once: it locks Storage tables, which pauses every bucket's Storage traffic until the transaction ends.
site_level_channelsDrops the company roster, and turns every company-level artifact back into an unresolved one with no project-undetermined item behind it.
admin_writesDrops category descriptions. Refuses if a retired category's name was added again.
contextualizeDrops each text's recorded parties and who filed each artifact, so a later backfill could overwrite an administrator's filing.
queue_and_artifact_writesRefuses while any captured artifact has a level with no ceiling, which is one set by hand or by an exclusion. Clear those first.
roster_backfillDrops every backfill request, pending or done. Roll the worker back first. Texts from a pending one stay unattributed and held until the person is added again.
category_sourceDrops who set each category. Corrections written meanwhile then read as model errors, so do not calibrate over them. Roll the project page back first.
resolve_uncategorized_partialRestores the prior whole-card category resolve. Items already split remain a closed answered item and an ordinary open continuing item; past Daybook split details remain unread. Roll the web app back first, or partial answers are refused.
reprocess_reviewDrops the record of carried work and pending category cards, while carried values, inherited corrections, and reprocessing Daybook events remain. It also leaves awaiting-review runs with proposed generations that cannot be decided. Decide every run first, and take the reprocessing worker and review page out of service before reversing it.

Scratch databases only. The earlier schema migrations (foundation, identity, projects_roster_typology, intake_deposit, artifacts, reference_data, reconciliation, review_queue, governance_and_monitors) drop tables that hold a Site's real records: the Daybook, principals, projects and rosters, Catalog rows that address the Deposit, human corrections, confirmed reconciliations, the Review Queue, break-glass records. Never run them against a live Site.

Two more are dangerous in the other direction. lock_down_data_api reverses to the vendor's default exposure, which re-opens every table to the public key over HTTP. authorization removes every access policy and the exposed read surface, leaving a database nobody can read through the Data API. Neither has a legitimate use against a live Site.

A reverse script that refuses says so with an error. Do the clearing step it names, then run it again. Reverting Govern's code does not undo what it did: rows already governed stay governed and need deliberate reprocessing.

After any deploy